[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:01 dist-test-slave-1x4g krb5kdc[3141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:01 dist-test-slave-1x4g krb5kdc[3141](info): set up 2 sockets
Aug 07 20:08:01 dist-test-slave-1x4g krb5kdc[3141](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 20:08:03.004678 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:03.131210 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33653
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:03.221312 3155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:03.221575 3155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:03.221616 3155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:03.222817 3155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:03.222888 3155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:03.222911 3155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:03.222934 3155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:03.222986 3155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:03.224814 3155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:03.225265 3155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:03.225574 3155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:03.233740 3163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.234543 3160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.234735 3161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:03.235641 3155 server_base.cc:1034] running on GCE node
I20260807 20:08:03.236001 3155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:03.236430 3155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:03.237622 3155 hybrid_clock.cc:648] HybridClock initialized: now 1786133283237595 us; error 75 us; skew 500 ppm
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:03.240900 3155 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:03.241647 3155 webserver.cc:460] Webserver started at http://127.3.15.190:46791/ using document root <none> and password file <none>
I20260807 20:08:03.241921 3155 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:03.241976 3155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:03.242110 3155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:03.243506 3155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "39f8962fddd14564a572c5a209c4c54f"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "92b3400ed73c9d7819251c71deb2da17"
server_key_iv: "ea7f4b9cda0c7a6a301f116bc8446bca"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.244095 3155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "39f8962fddd14564a572c5a209c4c54f"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "92b3400ed73c9d7819251c71deb2da17"
server_key_iv: "ea7f4b9cda0c7a6a301f116bc8446bca"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.246127 3155 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260807 20:08:03.247190 3170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.247468 3155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:03.247581 3155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "39f8962fddd14564a572c5a209c4c54f"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "92b3400ed73c9d7819251c71deb2da17"
server_key_iv: "ea7f4b9cda0c7a6a301f116bc8446bca"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.247680 3155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:03.261929 3155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:03.264119 3155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:03.264336 3155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:03.270066 3155 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:33653
I20260807 20:08:03.270205 3225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:33653 every 8 connection(s)
I20260807 20:08:03.270710 3155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:03.272305 3226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:03.275676 3226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f: Bootstrap starting.
I20260807 20:08:03.276712 3226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:03.277114 3226 log.cc:824] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:03.277266 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3155
I20260807 20:08:03.277382 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:03.278002 3226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f: No bootstrap required, opened a new log
I20260807 20:08:03.278072 3134 external_mini_cluster.cc:1428] Setting key b8996a24fd16b752330f365bf498f03d
I20260807 20:08:03.280398 3226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f8962fddd14564a572c5a209c4c54f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33653 } }
I20260807 20:08:03.280678 3226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:03.280827 3226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f8962fddd14564a572c5a209c4c54f, State: Initialized, Role: FOLLOWER
I20260807 20:08:03.281040 3226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f8962fddd14564a572c5a209c4c54f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33653 } }
I20260807 20:08:03.281198 3226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:03.281250 3226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:03.281303 3226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:03.282404 3226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f8962fddd14564a572c5a209c4c54f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33653 } }
I20260807 20:08:03.282642 3226 leader_election.cc:302] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39f8962fddd14564a572c5a209c4c54f; no voters:
I20260807 20:08:03.282919 3226 leader_election.cc:288] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:03.282945 3232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:03.283337 3226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:03.285259 3232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [term 1 LEADER]: Becoming Leader. State: Replica: 39f8962fddd14564a572c5a209c4c54f, State: Running, Role: LEADER
I20260807 20:08:03.285405 3232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f8962fddd14564a572c5a209c4c54f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33653 } }
I20260807 20:08:03.286211 3233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39f8962fddd14564a572c5a209c4c54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f8962fddd14564a572c5a209c4c54f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33653 } } }
I20260807 20:08:03.286299 3233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:03.285989 3244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39f8962fddd14564a572c5a209c4c54f. Latest consensus state: current_term: 1 leader_uuid: "39f8962fddd14564a572c5a209c4c54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f8962fddd14564a572c5a209c4c54f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33653 } } }
I20260807 20:08:03.286737 3244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f [sys.catalog]: This master's current role is: LEADER
W20260807 20:08:03.287621 3247 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:08:03.287690 3247 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:08:03.287792 3248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:03.288177 3248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:03.290746 3248 catalog_manager.cc:1347] Generated new cluster ID: e95a4c8673704dffb309c34ca0676899
I20260807 20:08:03.290804 3248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:03.298022 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.279388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.3.15.190:33653)
0807 20:08:03.279726 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:03.279738 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:03.279935 (+ 197us) server_negotiation.cc:407] Connection header received
0807 20:08:03.282198 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:03.282204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:03.282289 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:03.282441 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:03.284676 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.286302 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.288091 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.288659 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.288829 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.293595 (+ 4766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:03.293600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:03.293600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:03.293613 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:03.295632 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.296172 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.296175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.296176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.296289 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.296630 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.296631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.296631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.296966 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:03.297120 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:03.297590 (+ 470us) server_negotiation.cc:299] Negotiation successful
0807 20:08:03.297772 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260807 20:08:03.298928 3248 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:03.299697 3248 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:03.304461 3248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39f8962fddd14564a572c5a209c4c54f: Generated new TSK 0
I20260807 20:08:03.304641 3248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:03.334034 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33653
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:03.440188 3254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:03.440416 3254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:03.440459 3254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:03.441653 3254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:03.441717 3254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:03.441766 3254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:03.443198 3254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33653
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:03.443554 3254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:03.443789 3254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:03.449432 3262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.449469 3260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.449627 3259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:03.449700 3254 server_base.cc:1034] running on GCE node
I20260807 20:08:03.449944 3254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:03.450191 3254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:03.451357 3254 hybrid_clock.cc:648] HybridClock initialized: now 1786133283451325 us; error 41 us; skew 500 ppm
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:03.454210 3254 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:03.454636 3254 webserver.cc:460] Webserver started at http://127.3.15.129:40307/ using document root <none> and password file <none>
I20260807 20:08:03.454867 3254 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:03.454936 3254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:03.455089 3254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:03.456152 3254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "39d3fd8734284fd58bcf318f0b3696fe"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "67f8e4d45064e0c0ee093aa8043e4a95"
server_key_iv: "e80efa7627750136daf80c0970e853c3"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.456532 3254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "39d3fd8734284fd58bcf318f0b3696fe"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "67f8e4d45064e0c0ee093aa8043e4a95"
server_key_iv: "e80efa7627750136daf80c0970e853c3"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.458292 3254 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 20:08:03.459268 3269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.459450 3254 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:03.459566 3254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "39d3fd8734284fd58bcf318f0b3696fe"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "67f8e4d45064e0c0ee093aa8043e4a95"
server_key_iv: "e80efa7627750136daf80c0970e853c3"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.459648 3254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:03.491266 3254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:03.493505 3254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:03.493677 3254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:03.494081 3254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:03.494462 3254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:03.494570 3254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.494614 3254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:03.494645 3254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.502267 3254 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40075
I20260807 20:08:03.502462 3385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40075 every 8 connection(s)
I20260807 20:08:03.502708 3254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:03.510536 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3254
I20260807 20:08:03.510644 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:03.510823 3134 external_mini_cluster.cc:1428] Setting key 4dd2cefe7a4ecaeac42310822e1460bf
I20260807 20:08:03.513300 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.503738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:42697 (local address 127.3.15.190:33653)
0807 20:08:03.503801 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:03.503803 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:03.504003 (+ 200us) server_negotiation.cc:407] Connection header received
0807 20:08:03.504492 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:03.504493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:03.504507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:03.504564 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:03.505742 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.506258 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.507273 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.507466 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.507518 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.509964 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:03.509966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:03.509966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:03.509973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:03.511700 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.512244 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.512245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.512245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.512298 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.512580 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.512581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.512581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.512683 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:03.512732 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:03.513149 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 20:08:03.513233 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:08:03.513536 3388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.503705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33653 (local address 127.3.15.129:42697)
0807 20:08:03.503865 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:03.503915 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:03.504402 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:03.504621 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:03.504624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:03.505343 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:03.505638 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:03.505646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.506345 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.506346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:03.507195 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:03.507198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.507540 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.507541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:03.507642 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.508088 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:03.508097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:03.509871 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:03.511779 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:03.511786 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:03.511789 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:03.512165 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:03.512363 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:03.512364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:03.512364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:03.512516 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:03.512747 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:03.512754 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:03.512955 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 20:08:03.513076 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 20:08:03.513262 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260807 20:08:03.514195 3386 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33653
I20260807 20:08:03.514317 3386 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:03.514537 3386 heartbeater.cc:511] Master 127.3.15.190:33653 requested a full tablet report, sending...
I20260807 20:08:03.515493 3188 ts_manager.cc:194] Registered new tserver with Master: 39d3fd8734284fd58bcf318f0b3696fe (127.3.15.129:40075)
I20260807 20:08:03.516605 3188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:42697
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:03.541864 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33653
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:03.632609 3392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:03.632925 3392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:03.632989 3392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:03.634161 3392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:03.634205 3392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:03.634233 3392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:03.635721 3392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33653
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:03.635977 3392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:03.636216 3392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:03.641382 3397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:03.641398 3392 server_base.cc:1034] running on GCE node
W20260807 20:08:03.641333 3398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.641346 3400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:03.641754 3392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:03.642007 3392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:03.643174 3392 hybrid_clock.cc:648] HybridClock initialized: now 1786133283643170 us; error 50 us; skew 500 ppm
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:03.646001 3392 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:03.646481 3392 webserver.cc:460] Webserver started at http://127.3.15.130:44597/ using document root <none> and password file <none>
I20260807 20:08:03.646716 3392 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:03.646787 3392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:03.646946 3392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:03.648073 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "cb8f5fc06d5b45c3b6bae3e3add34172"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "f851979265d354852c12904c443a5de2"
server_key_iv: "884059bd4bd7f07e97c674077dae2188"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.648459 3392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "cb8f5fc06d5b45c3b6bae3e3add34172"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "f851979265d354852c12904c443a5de2"
server_key_iv: "884059bd4bd7f07e97c674077dae2188"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.650007 3392 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:03.650859 3407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.651033 3392 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:03.651165 3392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "cb8f5fc06d5b45c3b6bae3e3add34172"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "f851979265d354852c12904c443a5de2"
server_key_iv: "884059bd4bd7f07e97c674077dae2188"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.651283 3392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:03.674898 3392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:03.677014 3392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:03.677202 3392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:03.677465 3392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:03.677842 3392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:03.677898 3392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.677942 3392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:03.677970 3392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.685140 3392 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:43145
I20260807 20:08:03.685243 3527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:43145 every 8 connection(s)
I20260807 20:08:03.685563 3392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:03.687942 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3392
I20260807 20:08:03.688016 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:03.688192 3134 external_mini_cluster.cc:1428] Setting key d27bbdb84ff97eaf0638ba666e1077c8
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:03.695672 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.686319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:40833 (local address 127.3.15.190:33653)
0807 20:08:03.686374 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:03.686378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:03.686599 (+ 221us) server_negotiation.cc:407] Connection header received
0807 20:08:03.687108 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:03.687109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:03.687121 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:03.687175 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:03.688162 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.688905 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.689911 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.690130 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.690219 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.692678 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:03.692680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:03.692680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:03.692688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:03.694234 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.694664 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.694665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.694666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.694734 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.694993 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.694994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.694995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.695119 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:03.695196 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:03.695480 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 20:08:03.695580 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:08:03.695874 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.686303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33653 (local address 127.3.15.130:40833)
0807 20:08:03.686468 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:03.686506 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:03.686987 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:03.687229 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:03.687232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:03.687806 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:03.688054 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:03.688061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.689001 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.689002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:03.689813 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:03.689816 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.690229 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.690231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:03.690364 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.690928 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:03.690938 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:03.692577 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:03.694339 (+ 1762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:03.694346 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:03.694349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:03.694583 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:03.694804 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:03.694804 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:03.694805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:03.694922 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:03.695189 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:03.695196 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:03.695344 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 20:08:03.695458 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 20:08:03.695624 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260807 20:08:03.696408 3528 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33653
I20260807 20:08:03.696524 3528 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:03.696768 3528 heartbeater.cc:511] Master 127.3.15.190:33653 requested a full tablet report, sending...
I20260807 20:08:03.697254 3188 ts_manager.cc:194] Registered new tserver with Master: cb8f5fc06d5b45c3b6bae3e3add34172 (127.3.15.130:43145)
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
I20260807 20:08:03.697799 3188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:40833
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:03.717567 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33653
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:03.805001 3533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:03.805241 3533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:03.805265 3533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:03.806505 3533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:03.806550 3533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:03.806578 3533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:03.808035 3533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33653
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:03.808280 3533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:03.808522 3533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:03.813556 3539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.813570 3541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:03.813855 3538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:03.813818 3533 server_base.cc:1034] running on GCE node
I20260807 20:08:03.814064 3533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:03.814322 3533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:03.815495 3533 hybrid_clock.cc:648] HybridClock initialized: now 1786133283815480 us; error 29 us; skew 500 ppm
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:03.818277 3533 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:03.818781 3533 webserver.cc:460] Webserver started at http://127.3.15.131:40327/ using document root <none> and password file <none>
I20260807 20:08:03.819028 3533 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:03.819073 3533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:03.819271 3533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:03.820348 3533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "cad6d165162643d5bf54d6dc91cb0fc7"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "f2d4b061b40ccec4b3db3681692a3d17"
server_key_iv: "157c8dfa42259078dca7fbc235902679"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.820731 3533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "cad6d165162643d5bf54d6dc91cb0fc7"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "f2d4b061b40ccec4b3db3681692a3d17"
server_key_iv: "157c8dfa42259078dca7fbc235902679"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.822105 3533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:03.822846 3549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.823014 3533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:03.823102 3533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "cad6d165162643d5bf54d6dc91cb0fc7"
format_stamp: "Formatted at 2026-08-07 20:08:03 on dist-test-slave-1x4g"
server_key: "f2d4b061b40ccec4b3db3681692a3d17"
server_key_iv: "157c8dfa42259078dca7fbc235902679"
server_key_version: "encryptionkey@0"
I20260807 20:08:03.823158 3533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:03.847237 3533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:03.849699 3533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:03.849812 3533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:03.850014 3533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:03.850275 3533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:03.850307 3533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.850358 3533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:03.850394 3533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:03.857275 3533 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:39555
I20260807 20:08:03.857323 3666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:39555 every 8 connection(s)
I20260807 20:08:03.857646 3533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:03.863332 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3533
I20260807 20:08:03.863425 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:03.863595 3134 external_mini_cluster.cc:1428] Setting key d8fe9a4b9e26e4ee99f11cab4300173d
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:03.867918 3229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.858367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:50891 (local address 127.3.15.190:33653)
0807 20:08:03.858449 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:03.858452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:03.858619 (+ 167us) server_negotiation.cc:407] Connection header received
0807 20:08:03.859142 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:03.859143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:03.859153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:03.859212 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:03.860257 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.860878 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.861939 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.862140 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.862204 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.864825 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:03.864827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:03.864827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:03.864835 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:03.866393 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.866859 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.866868 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.866869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.866936 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.867222 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.867223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.867223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.867349 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:03.867437 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:03.867722 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 20:08:03.867807 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 20:08:03.868101 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.858325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33653 (local address 127.3.15.131:50891)
0807 20:08:03.858500 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:03.858537 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:03.859028 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:03.859292 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:03.859296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:03.859886 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:03.860135 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:03.860140 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.860961 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.860962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:03.861803 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:03.861806 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.862220 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.862221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:03.862367 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.862852 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:03.862861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:03.864677 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:03.866503 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:03.866511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:03.866513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:03.866756 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:03.867011 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:03.867012 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:03.867012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:03.867140 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:03.867426 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:03.867433 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:03.867579 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 20:08:03.867697 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 20:08:03.867860 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260807 20:08:03.868528 3667 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33653
I20260807 20:08:03.868614 3667 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:03.868892 3667 heartbeater.cc:511] Master 127.3.15.190:33653 requested a full tablet report, sending...
I20260807 20:08:03.869282 3188 ts_manager.cc:194] Registered new tserver with Master: cad6d165162643d5bf54d6dc91cb0fc7 (127.3.15.131:39555)
I20260807 20:08:03.869663 3188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:50891
I20260807 20:08:03.875764 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:03.900382 3675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.892595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.3.15.129:40075)
0807 20:08:03.892820 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:03.892826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:03.892838 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:08:03.892873 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:03.892875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:03.892889 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:03.892954 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:03.893571 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.894301 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.895063 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.895378 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.895433 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.897619 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:03.897627 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:03.897628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:03.897634 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:03.899242 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.899589 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.899592 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.899593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.899650 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.899819 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.899820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.899820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.899950 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:03.900068 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:03.900198 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:08:03.900263 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":102,"threads_started":1}
W20260807 20:08:03.900945 3298 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45006
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:08:03 dist-test-slave-1x4g krb5kdc[3141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133283, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:03.925940 3675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:03.918568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45014 (local address 127.3.15.129:40075)
0807 20:08:03.918643 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:03.918645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:03.918825 (+ 180us) server_negotiation.cc:407] Connection header received
0807 20:08:03.918928 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:03.918929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:03.918940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:03.919011 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:03.919604 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.920102 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.920923 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:03.921134 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:03.921213 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:03.923491 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:03.923494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:03.923494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:03.923501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:03.924732 (+ 1231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.925191 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.925192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.925193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.925256 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:03.925462 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:03.925463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:03.925464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:03.925569 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:03.925707 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:03.925803 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 20:08:03.925871 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:03.926697 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3254
I20260807 20:08:03.933346 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3392
I20260807 20:08:03.939003 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3533
I20260807 20:08:03.945079 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3155
2026-08-07T20:08:03Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2030 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:04 dist-test-slave-1x4g krb5kdc[3682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:04 dist-test-slave-1x4g krb5kdc[3682](info): set up 2 sockets
Aug 07 20:08:04 dist-test-slave-1x4g krb5kdc[3682](info): commencing operation
krb5kdc: starting...
W20260807 20:08:05.017809 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:05.141598 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39511
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39511
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:05.224591 3696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:05.224829 3696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:05.224853 3696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:05.225971 3696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:05.226008 3696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:05.226023 3696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:05.226035 3696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:05.226047 3696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:05.227564 3696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:05.227769 3696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:05.227950 3696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:05.232997 3704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.232987 3701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.232987 3702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:05.233282 3696 server_base.cc:1034] running on GCE node
I20260807 20:08:05.233573 3696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:05.233942 3696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:05.235167 3696 hybrid_clock.cc:648] HybridClock initialized: now 1786133285235155 us; error 58 us; skew 500 ppm
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:05.237932 3696 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:05.238389 3696 webserver.cc:460] Webserver started at http://127.3.15.190:42823/ using document root <none> and password file <none>
I20260807 20:08:05.238634 3696 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:05.238704 3696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:05.238860 3696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:05.240015 3696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "40e2645d4239406887e543292b36ade6"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "2eac3ce509ebdc1447c6d65b9f03f035"
server_key_iv: "bfb69b55761d9e843b74c5c085e4ca8a"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.240404 3696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "40e2645d4239406887e543292b36ade6"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "2eac3ce509ebdc1447c6d65b9f03f035"
server_key_iv: "bfb69b55761d9e843b74c5c085e4ca8a"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.241919 3696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:05.242678 3713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.242890 3696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:05.243063 3696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "40e2645d4239406887e543292b36ade6"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "2eac3ce509ebdc1447c6d65b9f03f035"
server_key_iv: "bfb69b55761d9e843b74c5c085e4ca8a"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.243150 3696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:05.269614 3696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:05.271837 3696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:05.272033 3696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:05.276135 3696 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39511
I20260807 20:08:05.276211 3767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39511 every 8 connection(s)
I20260807 20:08:05.276574 3696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:05.277065 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3696
I20260807 20:08:05.277218 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:05.277459 3134 external_mini_cluster.cc:1428] Setting key 048616cf23c1f63e6decfc71b529da1f
I20260807 20:08:05.278004 3768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:05.280743 3768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6: Bootstrap starting.
I20260807 20:08:05.281399 3768 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:05.281708 3768 log.cc:824] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:05.282486 3768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6: No bootstrap required, opened a new log
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:05.284596 3768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e2645d4239406887e543292b36ade6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39511 } }
I20260807 20:08:05.284824 3768 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:05.284883 3768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e2645d4239406887e543292b36ade6, State: Initialized, Role: FOLLOWER
I20260807 20:08:05.285030 3768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e2645d4239406887e543292b36ade6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39511 } }
I20260807 20:08:05.285128 3768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:05.285174 3768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:05.285229 3768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:05.285840 3768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e2645d4239406887e543292b36ade6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39511 } }
I20260807 20:08:05.285929 3768 leader_election.cc:302] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40e2645d4239406887e543292b36ade6; no voters:
I20260807 20:08:05.286132 3768 leader_election.cc:288] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:05.286154 3773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:05.286295 3773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [term 1 LEADER]: Becoming Leader. State: Replica: 40e2645d4239406887e543292b36ade6, State: Running, Role: LEADER
I20260807 20:08:05.286394 3773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e2645d4239406887e543292b36ade6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39511 } }
I20260807 20:08:05.286432 3768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:05.286733 3775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40e2645d4239406887e543292b36ade6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e2645d4239406887e543292b36ade6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39511 } } }
I20260807 20:08:05.287003 3775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:05.286906 3774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40e2645d4239406887e543292b36ade6. Latest consensus state: current_term: 1 leader_uuid: "40e2645d4239406887e543292b36ade6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40e2645d4239406887e543292b36ade6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39511 } } }
I20260807 20:08:05.287303 3774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:05.287310 3783 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:05.287949 3783 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:05.288125 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.278058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.3.15.190:39511)
0807 20:08:05.278273 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.278281 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.278308 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:08:05.278751 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.278758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.278803 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.278964 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.279594 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.280527 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.281411 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.281953 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.282074 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.284386 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.284390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.284390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.284401 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.286254 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.286686 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.286689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.286690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.286798 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.287066 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.287067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.287067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.287399 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.287562 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.287676 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.287826 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260807 20:08:05.289837 3783 catalog_manager.cc:1347] Generated new cluster ID: 0fa312a383fe4feb84e239612188f44e
I20260807 20:08:05.289898 3783 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:05.300732 3783 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:05.301481 3783 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:05.304427 3783 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40e2645d4239406887e543292b36ade6: Generated new TSK 0
I20260807 20:08:05.304600 3783 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:05.332242 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39511
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:05.420199 3795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:05.420444 3795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:05.420485 3795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:05.421725 3795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:05.421797 3795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:05.421840 3795 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:05.423578 3795 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39511
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:05.423854 3795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:05.424095 3795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:05.429675 3801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.429795 3803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.429752 3800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:05.429852 3795 server_base.cc:1034] running on GCE node
I20260807 20:08:05.430133 3795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:05.430426 3795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:05.431604 3795 hybrid_clock.cc:648] HybridClock initialized: now 1786133285431593 us; error 31 us; skew 500 ppm
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:05.438089 3795 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:05.438591 3795 webserver.cc:460] Webserver started at http://127.3.15.129:33515/ using document root <none> and password file <none>
I20260807 20:08:05.438822 3795 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:05.438890 3795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:05.439050 3795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:05.440289 3795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "390e909fff7c442b8118e59043ade366"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "fa8dcd464f494c579fd8b71bf4704575"
server_key_iv: "d58e6628ebed0c079fb0da96855fd41b"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.440711 3795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "390e909fff7c442b8118e59043ade366"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "fa8dcd464f494c579fd8b71bf4704575"
server_key_iv: "d58e6628ebed0c079fb0da96855fd41b"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.442152 3795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:05.442929 3810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.443105 3795 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:05.443214 3795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "390e909fff7c442b8118e59043ade366"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "fa8dcd464f494c579fd8b71bf4704575"
server_key_iv: "d58e6628ebed0c079fb0da96855fd41b"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.443295 3795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:05.459122 3795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:05.461442 3795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:05.461623 3795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:05.461881 3795 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:05.462261 3795 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:05.462318 3795 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.462352 3795 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:05.462384 3795 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.469661 3795 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:45491
I20260807 20:08:05.469722 3927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:45491 every 8 connection(s)
I20260807 20:08:05.470098 3795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:05.478559 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3795
I20260807 20:08:05.478669 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:05.478833 3134 external_mini_cluster.cc:1428] Setting key d0a7e76c6563667db5f29d31de5a6f5f
I20260807 20:08:05.480017 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.470955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:33059 (local address 127.3.15.190:39511)
0807 20:08:05.471023 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.471026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.471213 (+ 187us) server_negotiation.cc:407] Connection header received
0807 20:08:05.472335 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.472337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.472349 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.472393 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.473114 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.473626 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.474682 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.474897 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.474989 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.477147 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.477148 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.477149 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.477155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.478409 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.478884 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.478886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.478886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.479007 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.479303 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.479305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.479305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.479442 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.479522 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.479817 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.479926 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:08:05.480203 3930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.470855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39511 (local address 127.3.15.129:33059)
0807 20:08:05.471070 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.471128 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.472214 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.472476 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.472480 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.472739 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:05.473008 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.473013 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.473722 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.473724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.474588 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.474590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.475001 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.475002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.475137 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.475606 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:05.475614 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:05.477044 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:05.478497 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:05.478504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:05.478506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:05.478809 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:05.479076 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:05.479077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:05.479077 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:05.479225 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:05.479512 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:05.479520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:05.479672 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.479783 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.479952 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":111,"threads_started":1}
I20260807 20:08:05.480707 3928 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39511
I20260807 20:08:05.480850 3928 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:05.481024 3928 heartbeater.cc:511] Master 127.3.15.190:39511 requested a full tablet report, sending...
I20260807 20:08:05.481567 3732 ts_manager.cc:194] Registered new tserver with Master: 390e909fff7c442b8118e59043ade366 (127.3.15.129:45491)
I20260807 20:08:05.482196 3732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:33059
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:05.505337 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39511
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:05.588387 3933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:05.588598 3933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:05.588622 3933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:05.589783 3933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:05.589824 3933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:05.589850 3933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:05.591341 3933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39511
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:05.591594 3933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:05.591825 3933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:05.596716 3938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:05.596797 3933 server_base.cc:1034] running on GCE node
W20260807 20:08:05.596709 3939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.596709 3941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:05.597116 3933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:05.597317 3933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:05.598515 3933 hybrid_clock.cc:648] HybridClock initialized: now 1786133285598493 us; error 44 us; skew 500 ppm
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:05.601058 3933 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:05.601574 3933 webserver.cc:460] Webserver started at http://127.3.15.130:45367/ using document root <none> and password file <none>
I20260807 20:08:05.601826 3933 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:05.601897 3933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:05.602049 3933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:05.603083 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "3850212a2e1f49b3ab808861c4da8678"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "9876394bb371afc94212bf3342d6c0fa"
server_key_iv: "36dad1943ff842540e11a72d6dfb8d5c"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.603457 3933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "3850212a2e1f49b3ab808861c4da8678"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "9876394bb371afc94212bf3342d6c0fa"
server_key_iv: "36dad1943ff842540e11a72d6dfb8d5c"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.604812 3933 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:05.606542 3948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.606750 3933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:05.606853 3933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "3850212a2e1f49b3ab808861c4da8678"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "9876394bb371afc94212bf3342d6c0fa"
server_key_iv: "36dad1943ff842540e11a72d6dfb8d5c"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.606938 3933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:05.631502 3933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:05.633430 3933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:05.633595 3933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:05.633836 3933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:05.634167 3933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:05.634223 3933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.634265 3933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:05.634294 3933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.640187 3933 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:33595
I20260807 20:08:05.640235 4065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:33595 every 8 connection(s)
I20260807 20:08:05.640579 3933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:05.650277 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.641320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:45207 (local address 127.3.15.190:39511)
0807 20:08:05.641372 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.641374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.641609 (+ 235us) server_negotiation.cc:407] Connection header received
0807 20:08:05.642635 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.642637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.642651 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.642721 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.643393 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.643904 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.645003 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.645208 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.645295 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.647677 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.647679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.647679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.647686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.649045 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.649455 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.649456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.649456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.649507 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.649752 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.649753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.649753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.649851 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.649902 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.650152 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.650213 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:05.650507 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.641286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39511 (local address 127.3.15.130:45207)
0807 20:08:05.641478 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.641515 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.642491 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.642815 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.642821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.643086 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:05.643291 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.643295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.644008 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.644009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.644916 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.644918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.645308 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.645309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.645448 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.645991 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:05.646000 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:05.647586 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:05.649122 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:05.649130 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:05.649132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:05.649383 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:05.649570 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:05.649571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:05.649571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:05.649690 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:05.649910 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:05.649915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:05.650039 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.650126 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.650255 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260807 20:08:05.650658 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 3933
I20260807 20:08:05.650780 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:05.650943 3134 external_mini_cluster.cc:1428] Setting key b25c1361995b85e36838951968fcead0
I20260807 20:08:05.651190 4066 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39511
I20260807 20:08:05.651292 4066 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:05.651473 4066 heartbeater.cc:511] Master 127.3.15.190:39511 requested a full tablet report, sending...
I20260807 20:08:05.651937 3732 ts_manager.cc:194] Registered new tserver with Master: 3850212a2e1f49b3ab808861c4da8678 (127.3.15.130:33595)
I20260807 20:08:05.652343 3732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:45207
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:05.675500 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39511
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:05.759859 4071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:05.760067 4071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:05.760133 4071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:05.761328 4071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:05.761369 4071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:05.761415 4071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:05.762900 4071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39511
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:05.763085 4071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:05.763270 4071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:05.768215 4077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.768208 4076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:05.768208 4079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:05.768517 4071 server_base.cc:1034] running on GCE node
I20260807 20:08:05.768728 4071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:05.769034 4071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:05.770176 4071 hybrid_clock.cc:648] HybridClock initialized: now 1786133285770166 us; error 27 us; skew 500 ppm
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:05.772675 4071 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:05.773211 4071 webserver.cc:460] Webserver started at http://127.3.15.131:37341/ using document root <none> and password file <none>
I20260807 20:08:05.773416 4071 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:05.773458 4071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:05.773550 4071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:05.774575 4071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "28ed23ac983b4047aad1674c21e880c5"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "dd7989a1e3a2b90225b16c39f57eebe9"
server_key_iv: "068cfd4755fc7c09dbb84309d3988291"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.774886 4071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "28ed23ac983b4047aad1674c21e880c5"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "dd7989a1e3a2b90225b16c39f57eebe9"
server_key_iv: "068cfd4755fc7c09dbb84309d3988291"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.776219 4071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:05.776950 4086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.777133 4071 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:05.777236 4071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "28ed23ac983b4047aad1674c21e880c5"
format_stamp: "Formatted at 2026-08-07 20:08:05 on dist-test-slave-1x4g"
server_key: "dd7989a1e3a2b90225b16c39f57eebe9"
server_key_iv: "068cfd4755fc7c09dbb84309d3988291"
server_key_version: "encryptionkey@0"
I20260807 20:08:05.777320 4071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:05.812951 4071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:05.814842 4071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:05.815003 4071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:05.815261 4071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:05.815583 4071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:05.815640 4071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.815687 4071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:05.815721 4071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:05.821787 4071 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:46773
I20260807 20:08:05.821832 4204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:46773 every 8 connection(s)
I20260807 20:08:05.822170 4071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:05.831192 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 4071
I20260807 20:08:05.831305 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:05.831481 3134 external_mini_cluster.cc:1428] Setting key f753a38bc98893280f9b4613df54c1c3
I20260807 20:08:05.831707 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.822922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:59995 (local address 127.3.15.190:39511)
0807 20:08:05.822987 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.822990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.823209 (+ 219us) server_negotiation.cc:407] Connection header received
0807 20:08:05.824152 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.824153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.824164 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.824212 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.824891 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.825398 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.826432 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.826634 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.826715 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.828980 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.828981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.828981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.828989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.830274 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.830683 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.830684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.830685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.830751 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.831016 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.831017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.831017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.831145 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.831223 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.831528 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.831612 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:08:05.831902 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.822894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39511 (local address 127.3.15.131:59995)
0807 20:08:05.823085 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.823120 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.824054 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.824279 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.824283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.824540 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:05.824745 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.824749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.825502 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.825503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.826335 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.826337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.826723 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.826724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.826871 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.827442 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:05.827451 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:05.828883 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:05.830361 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:05.830368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:05.830371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:05.830604 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:05.830821 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:05.830821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:05.830822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:05.830939 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:05.831214 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:05.831221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:05.831381 (+ 160us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.831496 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.831669 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":109,"threads_started":1}
I20260807 20:08:05.832506 4205 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39511
I20260807 20:08:05.832615 4205 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:05.832763 4205 heartbeater.cc:511] Master 127.3.15.190:39511 requested a full tablet report, sending...
I20260807 20:08:05.833184 3732 ts_manager.cc:194] Registered new tserver with Master: 28ed23ac983b4047aad1674c21e880c5 (127.3.15.131:46773)
I20260807 20:08:05.833514 3732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:59995
I20260807 20:08:05.843243 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:08:05.851325 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.845266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.3.15.190:39511)
0807 20:08:05.845329 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.845331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.845440 (+ 109us) server_negotiation.cc:407] Connection header received
0807 20:08:05.845480 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.845480 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.845489 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.845540 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.846092 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.846742 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.847589 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.847774 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.847837 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.849009 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.849011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.849011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.849018 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.850282 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.850595 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.850596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.850596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.850655 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.850863 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.850864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.850865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.851027 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.851115 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.851202 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.851263 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:08:05.854068 3732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:05.855957 3732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:05.863529 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.859192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:33595 (local address 127.0.0.1:57080)
0807 20:08:05.859656 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.859668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.859727 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.859989 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.859990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.859992 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:05.860075 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.860077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.861429 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.861430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.862549 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.862551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.863219 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.863220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.863301 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.863314 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.863383 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.863442 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":45,"thread_start_us":84,"threads_started":1}
I20260807 20:08:05.863610 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.859187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46773 (local address 127.0.0.1:51474)
0807 20:08:05.859329 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.859370 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.859508 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.859977 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.859981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.859983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:05.860055 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.860057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.861137 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.861138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.862454 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.862457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.863351 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.863352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.863411 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.863414 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.863460 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.863492 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260807 20:08:05.863781 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.859551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47190 (local address 127.3.15.129:45491)
0807 20:08:05.859745 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.859750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.859908 (+ 158us) server_negotiation.cc:407] Connection header received
0807 20:08:05.859983 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.859985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.860001 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.860078 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:05.860302 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.861385 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.862506 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.863238 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.863312 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.863370 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.863658 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.863708 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":85,"threads_started":1}
I20260807 20:08:05.863880 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.859275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.3.15.131:46773)
0807 20:08:05.859763 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.859767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.859777 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:05.859813 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.859825 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.859843 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.860156 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:05.860252 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.861020 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.862546 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.863239 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.863343 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.863395 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.863743 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.863796 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":89,"threads_started":1}
I20260807 20:08:05.864022 4220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.859394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:45491 (local address 127.0.0.1:47190)
0807 20:08:05.859841 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.859849 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.859909 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.860168 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.860168 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.860170 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:05.860224 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.860226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.861476 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.861477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.862417 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.862419 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.863406 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.863407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.863481 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.863485 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.863535 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.863576 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":73,"threads_started":1}
I20260807 20:08:05.864024 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.859406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57080 (local address 127.3.15.130:33595)
0807 20:08:05.859638 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.859643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.859720 (+ 77us) server_negotiation.cc:407] Connection header received
0807 20:08:05.859828 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.859830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.859846 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.859939 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:05.860164 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.861320 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.862643 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.863131 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.863218 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.863280 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.863464 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.863513 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
I20260807 20:08:05.864748 4135 tablet_service.cc:1467] Processing CreateTablet for tablet a567654a10324a0b8b4c99deeef8ac09 (DEFAULT_TABLE table=test-table [id=f2f09ac4f5024325a28afd80ab527581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:05.864861 3859 tablet_service.cc:1467] Processing CreateTablet for tablet a567654a10324a0b8b4c99deeef8ac09 (DEFAULT_TABLE table=test-table [id=f2f09ac4f5024325a28afd80ab527581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:05.865186 4135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a567654a10324a0b8b4c99deeef8ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:05.865211 3859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a567654a10324a0b8b4c99deeef8ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:05.865206 3997 tablet_service.cc:1467] Processing CreateTablet for tablet a567654a10324a0b8b4c99deeef8ac09 (DEFAULT_TABLE table=test-table [id=f2f09ac4f5024325a28afd80ab527581]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:05.865515 3997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a567654a10324a0b8b4c99deeef8ac09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:05.867781 4223 tablet_bootstrap.cc:492] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5: Bootstrap starting.
I20260807 20:08:05.868469 4223 tablet_bootstrap.cc:654] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:05.868954 4225 tablet_bootstrap.cc:492] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: Bootstrap starting.
I20260807 20:08:05.869325 4223 log.cc:824] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:05.869639 4225 tablet_bootstrap.cc:654] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:05.870088 4223 tablet_bootstrap.cc:492] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5: No bootstrap required, opened a new log
I20260807 20:08:05.870149 4223 ts_tablet_manager.cc:1397] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:05.870108 4225 log.cc:824] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:05.870416 4224 tablet_bootstrap.cc:492] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: Bootstrap starting.
I20260807 20:08:05.870800 4225 tablet_bootstrap.cc:492] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: No bootstrap required, opened a new log
I20260807 20:08:05.870884 4225 ts_tablet_manager.cc:1397] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:05.871724 4223 raft_consensus.cc:348] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.871865 4223 raft_consensus.cc:374] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:05.871887 4223 raft_consensus.cc:729] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ed23ac983b4047aad1674c21e880c5, State: Initialized, Role: FOLLOWER
I20260807 20:08:05.872011 4223 consensus_queue.cc:260] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.872308 4223 ts_tablet_manager.cc:1428] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:05.872445 4205 heartbeater.cc:503] Master 127.3.15.190:39511 was elected leader, sending a full tablet report...
I20260807 20:08:05.872532 4225 raft_consensus.cc:348] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.872712 4225 raft_consensus.cc:374] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:05.872761 4225 raft_consensus.cc:729] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3850212a2e1f49b3ab808861c4da8678, State: Initialized, Role: FOLLOWER
I20260807 20:08:05.872928 4225 consensus_queue.cc:260] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.873380 4225 ts_tablet_manager.cc:1428] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 20:08:05.873428 4224 tablet_bootstrap.cc:654] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:05.873421 4066 heartbeater.cc:503] Master 127.3.15.190:39511 was elected leader, sending a full tablet report...
I20260807 20:08:05.873971 4224 log.cc:824] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:05.874769 4224 tablet_bootstrap.cc:492] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: No bootstrap required, opened a new log
I20260807 20:08:05.874825 4224 ts_tablet_manager.cc:1397] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 20:08:05.876137 4224 raft_consensus.cc:348] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.876271 4224 raft_consensus.cc:374] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:05.876292 4224 raft_consensus.cc:729] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 390e909fff7c442b8118e59043ade366, State: Initialized, Role: FOLLOWER
I20260807 20:08:05.876434 4224 consensus_queue.cc:260] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.876847 4224 ts_tablet_manager.cc:1428] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:05.876916 3928 heartbeater.cc:503] Master 127.3.15.190:39511 was elected leader, sending a full tablet report...
W20260807 20:08:05.891078 4067 tablet.cc:2369] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:05.927980 4229 raft_consensus.cc:482] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:05.928117 4229 raft_consensus.cc:504] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.928705 4229 leader_election.cc:288] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 28ed23ac983b4047aad1674c21e880c5 (127.3.15.131:46773), 390e909fff7c442b8118e59043ade366 (127.3.15.129:45491)
I20260807 20:08:05.931807 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.928914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46773 (local address 127.3.15.130:37713)
0807 20:08:05.928992 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.929009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.929073 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.929263 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.929264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.929267 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:05.929364 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.929367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.930112 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.930113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.930926 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.930928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.931529 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.931530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.931621 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.931625 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.931682 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.931713 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 20:08:05.931821 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.929037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:37713 (local address 127.3.15.131:46773)
0807 20:08:05.929110 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.929112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.929122 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:05.929151 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.929151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.929165 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.929220 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:05.929460 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.930021 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.931022 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.931443 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.931501 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.931508 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.931702 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.931750 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 20:08:05.931907 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.929146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:59197 (local address 127.3.15.129:45491)
0807 20:08:05.929219 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.929222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.929354 (+ 132us) server_negotiation.cc:407] Connection header received
0807 20:08:05.929384 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.929386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.929399 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.929457 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:05.929629 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.930164 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.931118 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.931525 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.931598 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.931606 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.931787 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.931832 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:08:05.931952 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.929041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:45491 (local address 127.3.15.130:59197)
0807 20:08:05.929231 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:05.929240 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:05.929343 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:05.929480 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:05.929481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:05.929483 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:05.929557 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.929558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.930245 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.930246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.931031 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:05.931034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.931614 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.931615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:05.931702 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.931707 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:05.931763 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:08:05.931803 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":123,"threads_started":1}
I20260807 20:08:05.932200 4157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a567654a10324a0b8b4c99deeef8ac09" candidate_uuid: "3850212a2e1f49b3ab808861c4da8678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ed23ac983b4047aad1674c21e880c5" is_pre_election: true
I20260807 20:08:05.932432 4157 raft_consensus.cc:2454] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3850212a2e1f49b3ab808861c4da8678 in term 0.
I20260807 20:08:05.932490 3881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a567654a10324a0b8b4c99deeef8ac09" candidate_uuid: "3850212a2e1f49b3ab808861c4da8678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390e909fff7c442b8118e59043ade366" is_pre_election: true
I20260807 20:08:05.932648 3881 raft_consensus.cc:2454] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3850212a2e1f49b3ab808861c4da8678 in term 0.
I20260807 20:08:05.932741 3950 leader_election.cc:302] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28ed23ac983b4047aad1674c21e880c5, 3850212a2e1f49b3ab808861c4da8678; no voters:
I20260807 20:08:05.932890 4229 raft_consensus.cc:2781] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:05.932951 4229 raft_consensus.cc:482] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:05.932978 4229 raft_consensus.cc:3037] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:05.933833 4229 raft_consensus.cc:504] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.933955 4229 leader_election.cc:288] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [CANDIDATE]: Term 1 election: Requested vote from peers 28ed23ac983b4047aad1674c21e880c5 (127.3.15.131:46773), 390e909fff7c442b8118e59043ade366 (127.3.15.129:45491)
I20260807 20:08:05.934172 4157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a567654a10324a0b8b4c99deeef8ac09" candidate_uuid: "3850212a2e1f49b3ab808861c4da8678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "28ed23ac983b4047aad1674c21e880c5"
I20260807 20:08:05.934156 3881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a567654a10324a0b8b4c99deeef8ac09" candidate_uuid: "3850212a2e1f49b3ab808861c4da8678" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "390e909fff7c442b8118e59043ade366"
I20260807 20:08:05.934248 4157 raft_consensus.cc:3037] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:05.934248 3881 raft_consensus.cc:3037] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:05.935328 3881 raft_consensus.cc:2454] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3850212a2e1f49b3ab808861c4da8678 in term 1.
I20260807 20:08:05.935529 3950 leader_election.cc:302] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3850212a2e1f49b3ab808861c4da8678, 390e909fff7c442b8118e59043ade366; no voters:
I20260807 20:08:05.935632 4229 raft_consensus.cc:2781] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:05.935681 4157 raft_consensus.cc:2454] T a567654a10324a0b8b4c99deeef8ac09 P 28ed23ac983b4047aad1674c21e880c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3850212a2e1f49b3ab808861c4da8678 in term 1.
I20260807 20:08:05.935801 4229 raft_consensus.cc:686] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [term 1 LEADER]: Becoming Leader. State: Replica: 3850212a2e1f49b3ab808861c4da8678, State: Running, Role: LEADER
I20260807 20:08:05.935904 4229 consensus_queue.cc:237] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } }
I20260807 20:08:05.936842 3732 catalog_manager.cc:5495] T a567654a10324a0b8b4c99deeef8ac09 P 3850212a2e1f49b3ab808861c4da8678 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3850212a2e1f49b3ab808861c4da8678 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "3850212a2e1f49b3ab808861c4da8678" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3850212a2e1f49b3ab808861c4da8678" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "28ed23ac983b4047aad1674c21e880c5" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "390e909fff7c442b8118e59043ade366" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 45491 } health_report { overall_health: UNKNOWN } } }
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:05.960920 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.953316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47192 (local address 127.3.15.129:45491)
0807 20:08:05.953414 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.953416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.953513 (+ 97us) server_negotiation.cc:407] Connection header received
0807 20:08:05.953556 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.953557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.953566 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.953614 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.954309 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.954779 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.955570 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.955747 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.955797 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.958336 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.958344 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.958345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.958351 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.959932 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.960259 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.960262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.960263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.960325 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.960480 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.960480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.960481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.960612 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.960713 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.960791 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.960854 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 20:08:05.970875 3929 tablet.cc:2369] T a567654a10324a0b8b4c99deeef8ac09 P 390e909fff7c442b8118e59043ade366: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:05.988968 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:05.980686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.3.15.129:45491)
0807 20:08:05.980752 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:05.980755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:05.980914 (+ 159us) server_negotiation.cc:407] Connection header received
0807 20:08:05.981048 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:05.981049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:05.981060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:05.981110 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:05.981672 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.982172 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.982921 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:05.983119 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:05.983186 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:05.985744 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:05.985746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:05.985746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:05.985754 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:05.987635 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.988053 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.988054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.988055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.988121 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:05.988329 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:05.988330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:05.988331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:05.988440 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:05.988538 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:05.988739 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 20:08:05.988863 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Aug 07 20:08:05 dist-test-slave-1x4g krb5kdc[3682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:08:06 dist-test-slave-1x4g krb5kdc[3682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:06.013330 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:06.006231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47206 (local address 127.3.15.129:45491)
0807 20:08:06.006294 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:06.006296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:06.006350 (+ 54us) server_negotiation.cc:407] Connection header received
0807 20:08:06.006421 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:06.006422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:06.006432 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:06.006489 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:06.007003 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:06.007466 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:06.008201 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:06.008388 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:06.008447 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:06.010953 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:06.010955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:06.010955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:06.010962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:06.012204 (+ 1242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:06.012532 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:06.012533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:06.012533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:06.012586 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:06.012913 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:06.012914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:06.012914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:06.013005 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:06.013076 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:06.013205 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 20:08:06.013262 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 20:08:06.014195 3838 tablet_service.cc:2908] Rejecting scan request for tablet a567654a10324a0b8b4c99deeef8ac09: Uninitialized: clean time has not yet been initialized
I20260807 20:08:06.015384 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3795
I20260807 20:08:06.022073 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3933
I20260807 20:08:06.029146 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 4071
I20260807 20:08:06.035068 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 3696
2026-08-07T20:08:06Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2089 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:06 dist-test-slave-1x4g krb5kdc[4249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:06 dist-test-slave-1x4g krb5kdc[4249](info): set up 2 sockets
Aug 07 20:08:06 dist-test-slave-1x4g krb5kdc[4249](info): commencing operation
krb5kdc: starting...
W20260807 20:08:07.101039 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:07.227252 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:36445
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:36445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:07.313019 4263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:07.313271 4263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:07.313313 4263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:07.314630 4263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:07.314698 4263 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 20:08:07.314728 4263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:07.314764 4263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:07.314795 4263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:07.314828 4263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:07.316370 4263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:36445
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:36445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:07.316677 4263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:07.316962 4263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:07.322045 4271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:07.322052 4268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:07.322149 4263 server_base.cc:1034] running on GCE node
W20260807 20:08:07.322250 4269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:07.322619 4263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:07.322911 4263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:07.324059 4263 hybrid_clock.cc:648] HybridClock initialized: now 1786133287324041 us; error 40 us; skew 500 ppm
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:07.326563 4263 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:07.326990 4263 webserver.cc:460] Webserver started at http://127.3.15.190:39543/ using document root <none> and password file <none>
I20260807 20:08:07.327246 4263 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:07.327314 4263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:07.327458 4263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:07.328532 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "8087b440d6674c198945a9bcfbc0fa96"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "355d157e42aca900b5fc97f5a5aad890"
server_key_iv: "d3dae948710e51f040cfb3314ad0fddc"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.329018 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "8087b440d6674c198945a9bcfbc0fa96"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "355d157e42aca900b5fc97f5a5aad890"
server_key_iv: "d3dae948710e51f040cfb3314ad0fddc"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.330405 4263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:07.331151 4278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.331339 4263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:07.331449 4263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "8087b440d6674c198945a9bcfbc0fa96"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "355d157e42aca900b5fc97f5a5aad890"
server_key_iv: "d3dae948710e51f040cfb3314ad0fddc"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.331522 4263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:07.343039 4263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:07.345184 4263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:07.345340 4263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:07.350260 4263 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:36445
I20260807 20:08:07.350369 4344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:36445 every 8 connection(s)
I20260807 20:08:07.350685 4263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:07.351369 4345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:07.352669 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 4263
I20260807 20:08:07.352777 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:07.353003 3134 external_mini_cluster.cc:1428] Setting key 1f773f546886832a9fd6bddf8f80f2ba
I20260807 20:08:07.354226 4345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: Bootstrap starting.
I20260807 20:08:07.354951 4345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:07.355298 4345 log.cc:824] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:07.356151 4345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: No bootstrap required, opened a new log
I20260807 20:08:07.358556 4345 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8087b440d6674c198945a9bcfbc0fa96" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36445 } }
I20260807 20:08:07.358695 4345 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:07.358718 4345 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8087b440d6674c198945a9bcfbc0fa96, State: Initialized, Role: FOLLOWER
I20260807 20:08:07.358870 4345 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8087b440d6674c198945a9bcfbc0fa96" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36445 } }
I20260807 20:08:07.358937 4345 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:07.358958 4345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:07.359026 4345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:07.359562 4345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8087b440d6674c198945a9bcfbc0fa96" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36445 } }
I20260807 20:08:07.359674 4345 leader_election.cc:302] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8087b440d6674c198945a9bcfbc0fa96; no voters:
I20260807 20:08:07.359840 4345 leader_election.cc:288] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:07.359896 4350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:07.360052 4350 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [term 1 LEADER]: Becoming Leader. State: Replica: 8087b440d6674c198945a9bcfbc0fa96, State: Running, Role: LEADER
I20260807 20:08:07.360108 4345 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:07.360183 4350 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8087b440d6674c198945a9bcfbc0fa96" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36445 } }
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:07.360556 4350 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8087b440d6674c198945a9bcfbc0fa96. Latest consensus state: current_term: 1 leader_uuid: "8087b440d6674c198945a9bcfbc0fa96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8087b440d6674c198945a9bcfbc0fa96" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36445 } } }
I20260807 20:08:07.360735 4350 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:07.361255 4351 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8087b440d6674c198945a9bcfbc0fa96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8087b440d6674c198945a9bcfbc0fa96" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36445 } } }
I20260807 20:08:07.361358 4351 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96 [sys.catalog]: This master's current role is: LEADER
W20260807 20:08:07.362319 4365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:08:07.362381 4365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:08:07.362509 4359 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:07.362746 4359 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:07.365020 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.353544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41438 (local address 127.3.15.190:36445)
0807 20:08:07.353755 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.353765 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.353813 (+ 48us) server_negotiation.cc:407] Connection header received
0807 20:08:07.354368 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.354377 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.354439 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.354611 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.355230 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.356582 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.357418 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.358075 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.358194 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.360987 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.360991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.360991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.361003 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.363114 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.363527 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.363530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.363531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.363637 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.363898 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.363900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.363900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.364217 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.364377 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.364553 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.364703 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":106,"threads_started":1}
I20260807 20:08:07.365218 4359 catalog_manager.cc:1347] Generated new cluster ID: fbdb1975cc834d088d5f0986b5f2e17b
I20260807 20:08:07.365259 4359 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:07.424890 4359 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:07.425643 4359 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:07.435019 4359 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: Generated new TSK 0
I20260807 20:08:07.435235 4359 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:07.475629 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36445
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:07.560729 4372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:07.561000 4372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:07.561048 4372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:07.562352 4372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:07.562407 4372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:07.562430 4372 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:07.562489 4372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:07.563968 4372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:36445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:07.564229 4372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:07.564469 4372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:07.569535 4380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:07.569542 4378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:07.569674 4372 server_base.cc:1034] running on GCE node
W20260807 20:08:07.569623 4377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:07.570055 4372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:07.570253 4372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:07.571395 4372 hybrid_clock.cc:648] HybridClock initialized: now 1786133287571382 us; error 30 us; skew 500 ppm
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:07.573956 4372 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:07.574371 4372 webserver.cc:460] Webserver started at http://127.3.15.129:32783/ using document root <none> and password file <none>
I20260807 20:08:07.574577 4372 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:07.574620 4372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:07.574710 4372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:07.575687 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "a4634369066e46e9a5050423dca462f9"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "96c79b5449ddced83439923d0b891e3e"
server_key_iv: "9901aaab6c802062c8c5df6c4ac0aa92"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.576064 4372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "a4634369066e46e9a5050423dca462f9"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "96c79b5449ddced83439923d0b891e3e"
server_key_iv: "9901aaab6c802062c8c5df6c4ac0aa92"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.577428 4372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 20:08:07.578130 4387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.578301 4372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:07.578408 4372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "a4634369066e46e9a5050423dca462f9"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "96c79b5449ddced83439923d0b891e3e"
server_key_iv: "9901aaab6c802062c8c5df6c4ac0aa92"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.578483 4372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:07.615218 4372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:07.617821 4372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:07.618006 4372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:07.618637 4372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:07.618708 4372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.618758 4372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:07.618788 4372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.626652 4372 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:43829
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:07.626724 4509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:43829 every 8 connection(s)
I20260807 20:08:07.627131 4372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:07.629884 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.619356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:45791 (local address 127.3.15.190:36445)
0807 20:08:07.619418 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.619421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.619634 (+ 213us) server_negotiation.cc:407] Connection header received
0807 20:08:07.620215 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.620217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.620233 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.620319 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.621392 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.622047 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.623712 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.624084 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.624167 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.627245 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.627246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.627247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.627255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.628553 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.629049 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.629050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.629050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.629116 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.629293 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.629293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.629293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.629387 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.629435 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.629752 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.629823 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:08:07.630071 4397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.619318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.3.15.129:45791)
0807 20:08:07.619497 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.619538 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.620093 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.620482 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.620487 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.620945 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:07.621259 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.621268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.622157 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.622158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.623596 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.623601 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.624188 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.624189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.624340 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.625435 (+ 1095us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:07.625444 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:07.627142 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:07.628629 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.628636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.628637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.628982 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.629165 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.629165 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.629165 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.629256 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.629468 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:07.629475 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:07.629599 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.629712 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.629832 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260807 20:08:07.630735 4510 heartbeater.cc:347] Connected to a master server at 127.3.15.190:36445
I20260807 20:08:07.630841 4510 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:07.631023 4510 heartbeater.cc:511] Master 127.3.15.190:36445 requested a full tablet report, sending...
I20260807 20:08:07.631618 4295 ts_manager.cc:194] Registered new tserver with Master: a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829)
I20260807 20:08:07.631953 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 4372
I20260807 20:08:07.632053 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:07.632254 3134 external_mini_cluster.cc:1428] Setting key bcedb17e63f7e4f21e13b81721a33414
I20260807 20:08:07.632588 4295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:45791
I20260807 20:08:07.637997 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.631865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.3.15.129:60015)
0807 20:08:07.631993 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.632002 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.632075 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.632219 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.632220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.632411 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:07.632467 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.632468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.633340 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.633341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.633852 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.633853 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.634303 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.634304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.634384 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.634841 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:07.634846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:07.635242 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:07.637075 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.637076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.637076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.637247 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.637461 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.637461 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.637461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.637511 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.637772 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:07.637773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:07.637809 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.637862 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.637936 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260807 20:08:07.638077 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.631936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:60015 (local address 127.3.15.190:36445)
0807 20:08:07.631976 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.631977 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.632051 (+ 74us) server_negotiation.cc:407] Connection header received
0807 20:08:07.632131 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.632132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.632143 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.632214 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.632527 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.633241 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.633995 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.634217 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.634301 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.635326 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.635327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.635327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.635334 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.636987 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.637323 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.637325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.637325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.637392 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.637569 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.637569 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.637570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.637706 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.637780 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.637888 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.637980 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:07.659332 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36445
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:07.742453 4520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:07.742679 4520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:07.742715 4520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:07.743856 4520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:07.743920 4520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:07.743964 4520 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:07.744021 4520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:07.745537 4520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:36445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:07.745792 4520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:07.746027 4520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:07.750980 4526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:07.751008 4525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:07.751070 4528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:07.751253 4520 server_base.cc:1034] running on GCE node
I20260807 20:08:07.751616 4520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:07.751850 4520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:07.753014 4520 hybrid_clock.cc:648] HybridClock initialized: now 1786133287753012 us; error 37 us; skew 500 ppm
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:07.755528 4520 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:07.755992 4520 webserver.cc:460] Webserver started at http://127.3.15.130:43921/ using document root <none> and password file <none>
I20260807 20:08:07.756238 4520 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:07.756313 4520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:07.756443 4520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:07.757508 4520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "c4aa133b5b4a45439ca14871b866ca5f"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "904c62b917001c0fdcfe9bae8455f858"
server_key_iv: "b15a9807417603a16280073e0e57af67"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.757814 4520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "c4aa133b5b4a45439ca14871b866ca5f"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "904c62b917001c0fdcfe9bae8455f858"
server_key_iv: "b15a9807417603a16280073e0e57af67"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.759187 4520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:07.759927 4536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.760118 4520 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:07.760212 4520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "c4aa133b5b4a45439ca14871b866ca5f"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "904c62b917001c0fdcfe9bae8455f858"
server_key_iv: "b15a9807417603a16280073e0e57af67"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.760272 4520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:07.768745 4520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:07.771049 4520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:07.771167 4520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:07.771616 4520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:07.771658 4520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.771682 4520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:07.771730 4520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.778873 4520 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:32947
I20260807 20:08:07.778951 4659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:32947 every 8 connection(s)
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:07.779346 4520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:07.782541 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.772136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56123 (local address 127.3.15.190:36445)
0807 20:08:07.772214 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.772216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.772389 (+ 173us) server_negotiation.cc:407] Connection header received
0807 20:08:07.773031 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.773032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.773045 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.773109 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.774093 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.774953 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.776667 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.776972 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.777031 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.779741 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.779743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.779743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.779751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.781245 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.781699 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.781700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.781700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.781754 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.781988 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.781989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.781989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.782093 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.782144 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.782377 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.782441 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 20:08:07.782740 4547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.772106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.3.15.130:56123)
0807 20:08:07.772279 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.772317 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.772882 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.773229 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.773234 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.773661 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:07.773978 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.773986 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.775061 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.775063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.776551 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.776555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.777058 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.777060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.777229 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.777829 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:07.777842 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:07.779610 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:07.781315 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.781320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.781322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.781629 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.781815 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.781816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.781816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.781916 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.782150 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:07.782155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:07.782266 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.782348 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.782469 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
I20260807 20:08:07.783254 4660 heartbeater.cc:347] Connected to a master server at 127.3.15.190:36445
I20260807 20:08:07.783335 4660 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:07.783522 4660 heartbeater.cc:511] Master 127.3.15.190:36445 requested a full tablet report, sending...
I20260807 20:08:07.783926 4296 ts_manager.cc:194] Registered new tserver with Master: c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947)
I20260807 20:08:07.784489 4296 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:56123
I20260807 20:08:07.785180 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 4520
I20260807 20:08:07.785281 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:07.785467 3134 external_mini_cluster.cc:1428] Setting key ba6648933d2a3625f6d4b184ae7fd272
I20260807 20:08:07.790210 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.784325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:43169 (local address 127.3.15.190:36445)
0807 20:08:07.784362 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.784364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.784429 (+ 65us) server_negotiation.cc:407] Connection header received
0807 20:08:07.784498 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.784498 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.784506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.784578 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.784991 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.785531 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.786357 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.786558 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.786628 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.787527 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.787528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.787528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.787535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.789071 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.789470 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.789471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.789472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.789528 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.789735 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.789736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.789736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.789871 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.789926 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.790089 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.790153 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:08:07.790325 4666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.784248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.3.15.130:43169)
0807 20:08:07.784375 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.784385 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.784434 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.784586 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.784587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.784841 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:07.784907 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.784913 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.785623 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.785624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.786270 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.786272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.786627 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.786628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.786701 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.787139 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:07.787144 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:07.787402 (+ 258us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:07.789171 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.789172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.789173 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.789395 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.789599 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.789600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.789600 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.789673 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.789937 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:07.789938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:07.789989 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.790061 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.790177 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":70,"threads_started":1}
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:07.812161 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36445
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:07.896168 4669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:07.896397 4669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:07.896436 4669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:07.897655 4669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:07.897720 4669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:07.897754 4669 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:07.897796 4669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:07.899221 4669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:36445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:07.899494 4669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:07.899724 4669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:07.904563 4674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:07.904574 4675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:07.904574 4678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:07.904963 4669 server_base.cc:1034] running on GCE node
I20260807 20:08:07.905182 4669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:07.905431 4669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:07.906584 4669 hybrid_clock.cc:648] HybridClock initialized: now 1786133287906570 us; error 25 us; skew 500 ppm
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:07.909129 4669 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:07.909600 4669 webserver.cc:460] Webserver started at http://127.3.15.131:38515/ using document root <none> and password file <none>
I20260807 20:08:07.909809 4669 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:07.909852 4669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:07.909955 4669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:07.910984 4669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "5a3bdc5f4d3d4414be1222ccd261bafe"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "e1351cbd1f32ed53b8d6b842ba0fe692"
server_key_iv: "c7a55d452532267e10370a0095d0d372"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.911299 4669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "5a3bdc5f4d3d4414be1222ccd261bafe"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "e1351cbd1f32ed53b8d6b842ba0fe692"
server_key_iv: "c7a55d452532267e10370a0095d0d372"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.912562 4669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:07.913414 4685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.913616 4669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:08:07.913728 4669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "5a3bdc5f4d3d4414be1222ccd261bafe"
format_stamp: "Formatted at 2026-08-07 20:08:07 on dist-test-slave-1x4g"
server_key: "e1351cbd1f32ed53b8d6b842ba0fe692"
server_key_iv: "c7a55d452532267e10370a0095d0d372"
server_key_version: "encryptionkey@0"
I20260807 20:08:07.913805 4669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:07.930552 4669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:07.932663 4669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:07.932883 4669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:07.933459 4669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:07.933521 4669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:07.933597 4669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:07.933629 4669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:07.941286 4669 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:42157
I20260807 20:08:07.941646 4669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:07.942094 4807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:42157 every 8 connection(s)
I20260807 20:08:07.943722 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.933869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:34901 (local address 127.3.15.190:36445)
0807 20:08:07.933955 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.933958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.934213 (+ 255us) server_negotiation.cc:407] Connection header received
0807 20:08:07.934722 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.934723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.934739 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.934830 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.935704 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.936548 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.937835 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.938053 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.938148 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.940734 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.940735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.940736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.940744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.942415 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.942896 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.942897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.942898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.942990 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.943226 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.943227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.943227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.943333 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.943385 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.943598 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.943660 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:08:07.943961 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.933868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.3.15.131:34901)
0807 20:08:07.934058 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.934111 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.934615 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.934918 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.934923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.935303 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:07.935595 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.935603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.936662 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.936663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.937743 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.937746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.938162 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.938163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.938308 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.938832 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:07.938842 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:07.940652 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:07.942520 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.942528 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.942531 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.942816 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.943057 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.943058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.943058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.943167 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.943389 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:07.943394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:07.943505 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.943584 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.943707 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":123,"threads_started":1}
I20260807 20:08:07.944478 4808 heartbeater.cc:347] Connected to a master server at 127.3.15.190:36445
I20260807 20:08:07.944552 4808 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:07.944739 4808 heartbeater.cc:511] Master 127.3.15.190:36445 requested a full tablet report, sending...
I20260807 20:08:07.945139 4295 ts_manager.cc:194] Registered new tserver with Master: 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131:42157)
I20260807 20:08:07.945528 4295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:34901
I20260807 20:08:07.947527 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 4669
I20260807 20:08:07.947633 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:07.947822 3134 external_mini_cluster.cc:1428] Setting key cb1f36973518c77992fc92689025ccb8
I20260807 20:08:07.948638 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:08:07.951196 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.945596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:46891 (local address 127.3.15.190:36445)
0807 20:08:07.945655 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.945657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.945739 (+ 82us) server_negotiation.cc:407] Connection header received
0807 20:08:07.945808 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.945809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.945816 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.945887 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.946186 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.946691 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.947403 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.947682 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.947743 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.948812 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.948813 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.948814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.948820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.950086 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.950435 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.950436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.950436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.950490 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.950700 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.950701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.950701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.950818 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.950887 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.951049 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.951113 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:08:07.951216 4814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.945532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.3.15.131:46891)
0807 20:08:07.945681 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.945693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.945753 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.945889 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.945892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.946044 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:07.946108 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.946109 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.946775 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.946776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.947283 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.947284 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.947762 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.947764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.947861 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.948320 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:07.948327 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:07.948714 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:07.950173 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.950174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.950174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.950348 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.950565 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:07.950566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:07.950566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:07.950625 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:07.950891 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:07.950892 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:07.950936 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.951012 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.951114 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:08:07 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:07.972970 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.966081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41454 (local address 127.3.15.190:36445)
0807 20:08:07.966144 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.966147 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.966176 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:08:07.966208 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.966209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.966219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.966273 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:07.966856 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.967360 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.968118 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.968310 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.968382 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.970608 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:07.970609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:07.970609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:07.970617 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:07.971776 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.972130 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.972131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.972132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.972190 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:07.972394 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:07.972395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:07.972395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:07.972505 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:07.972611 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.972748 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.972854 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:08:07.974431 4295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:07.976011 4295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:07.982489 4827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.978340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:43829 (local address 127.0.0.1:59622)
0807 20:08:07.978519 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.978566 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.978636 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.978822 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.978824 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.978827 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:07.978897 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.978899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.980052 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.980053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.981192 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.981194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.982170 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.982171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.982257 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.982272 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.982342 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.982394 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":82,"threads_started":1}
I20260807 20:08:07.982777 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.978359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42157 (local address 127.0.0.1:39764)
0807 20:08:07.978639 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.978651 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.978705 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.979079 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.979080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.979082 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:07.979121 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.979122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.980532 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.980533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.981617 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.981619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.982512 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.982513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.982578 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.982583 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.982644 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.982686 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":107,"threads_started":1}
I20260807 20:08:07.982913 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.979023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.3.15.130:32947)
0807 20:08:07.979384 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.979388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.979397 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:07.979436 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.979438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.979453 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.979531 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:07.979658 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.980664 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.981943 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.982557 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.982632 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.982690 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.982762 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.982803 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":93,"threads_started":1}
I20260807 20:08:07.982877 4829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.978573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:32947 (local address 127.0.0.1:43446)
0807 20:08:07.979216 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:07.979227 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:07.979275 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:07.979544 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:07.979545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:07.979547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:07.979585 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.979586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.980754 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.980756 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.981831 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:07.981834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.982634 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.982635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:07.982694 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.982698 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:07.982751 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:08:07.982798 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":69,"threads_started":1}
I20260807 20:08:07.983060 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.978452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59622 (local address 127.3.15.129:43829)
0807 20:08:07.978609 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.978615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.978626 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:07.978719 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.978721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.978736 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.978809 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:07.979002 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.979975 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.981287 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.982011 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.982080 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.982135 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.982897 (+ 762us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.982948 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260807 20:08:07.983335 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:07.978717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.3.15.131:42157)
0807 20:08:07.978896 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:07.978901 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:07.978914 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:08:07.978957 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:07.978959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:07.978975 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:07.979080 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:07.979269 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.980441 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.981706 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:07.982416 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:07.982495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:07.982557 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:07.983128 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 20:08:07.983215 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260807 20:08:07.983783 4434 tablet_service.cc:1467] Processing CreateTablet for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 (DEFAULT_TABLE table=test-table [id=d7cd1d4fc3a34cf5b0e767845b7f26a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:07.983989 4590 tablet_service.cc:1467] Processing CreateTablet for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 (DEFAULT_TABLE table=test-table [id=d7cd1d4fc3a34cf5b0e767845b7f26a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:07.984119 4740 tablet_service.cc:1467] Processing CreateTablet for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 (DEFAULT_TABLE table=test-table [id=d7cd1d4fc3a34cf5b0e767845b7f26a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:07.984231 4434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:07.984349 4590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:07.984428 4740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:07.987470 4832 tablet_bootstrap.cc:492] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Bootstrap starting.
I20260807 20:08:07.987815 4833 tablet_bootstrap.cc:492] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Bootstrap starting.
I20260807 20:08:07.988062 4834 tablet_bootstrap.cc:492] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Bootstrap starting.
I20260807 20:08:07.988404 4832 tablet_bootstrap.cc:654] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:07.989059 4832 log.cc:824] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:07.989329 4834 tablet_bootstrap.cc:654] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:07.989396 4833 tablet_bootstrap.cc:654] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:07.990185 4832 tablet_bootstrap.cc:492] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: No bootstrap required, opened a new log
I20260807 20:08:07.990298 4832 ts_tablet_manager.cc:1397] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:08:07.990338 4833 log.cc:824] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:07.990438 4834 log.cc:824] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:07.991360 4833 tablet_bootstrap.cc:492] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: No bootstrap required, opened a new log
I20260807 20:08:07.991372 4834 tablet_bootstrap.cc:492] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: No bootstrap required, opened a new log
I20260807 20:08:07.991438 4834 ts_tablet_manager.cc:1397] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:08:07.991447 4833 ts_tablet_manager.cc:1397] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260807 20:08:07.993040 4832 raft_consensus.cc:348] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:07.993202 4832 raft_consensus.cc:374] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:07.993263 4832 raft_consensus.cc:729] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4634369066e46e9a5050423dca462f9, State: Initialized, Role: FOLLOWER
I20260807 20:08:07.993391 4832 consensus_queue.cc:260] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:07.993824 4832 ts_tablet_manager.cc:1428] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:08:07.993873 4510 heartbeater.cc:503] Master 127.3.15.190:36445 was elected leader, sending a full tablet report...
I20260807 20:08:07.996225 4834 raft_consensus.cc:348] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:07.996225 4833 raft_consensus.cc:348] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:07.996390 4834 raft_consensus.cc:374] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:07.996390 4833 raft_consensus.cc:374] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:07.996429 4834 raft_consensus.cc:729] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4aa133b5b4a45439ca14871b866ca5f, State: Initialized, Role: FOLLOWER
I20260807 20:08:07.996429 4833 raft_consensus.cc:729] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3bdc5f4d3d4414be1222ccd261bafe, State: Initialized, Role: FOLLOWER
I20260807 20:08:07.996549 4833 consensus_queue.cc:260] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:07.996549 4834 consensus_queue.cc:260] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:07.996997 4834 ts_tablet_manager.cc:1428] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 20:08:07.996997 4833 ts_tablet_manager.cc:1428] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260807 20:08:07.997089 4808 heartbeater.cc:503] Master 127.3.15.190:36445 was elected leader, sending a full tablet report...
I20260807 20:08:07.997411 4660 heartbeater.cc:503] Master 127.3.15.190:36445 was elected leader, sending a full tablet report...
I20260807 20:08:08.021780 4840 raft_consensus.cc:482] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:08.021903 4840 raft_consensus.cc:504] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.022272 4840 leader_election.cc:288] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829), c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947)
I20260807 20:08:08.026090 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.022771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:53603 (local address 127.3.15.129:43829)
0807 20:08:08.022840 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.022843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.022865 (+ 22us) server_negotiation.cc:407] Connection header received
0807 20:08:08.022905 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.022906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.022919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.022990 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:08.023293 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.024136 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.025205 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.025720 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.025774 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.025781 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.026002 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.026036 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 20:08:08.026105 4694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.022636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:43829 (local address 127.3.15.131:53603)
0807 20:08:08.022710 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.022723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.022771 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.023011 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.023012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.023015 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:08.023197 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.023200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.024217 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.024218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.025103 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.025106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.025796 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.025797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.025885 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.025890 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.025980 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.026024 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 20:08:08.026547 4463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f1abdb1cc4d0b80a4bdeef4136fc7" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4634369066e46e9a5050423dca462f9" is_pre_election: true
I20260807 20:08:08.026798 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.023011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:50631 (local address 127.3.15.130:32947)
0807 20:08:08.023073 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.023075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.023105 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:08:08.023153 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.023154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.023173 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.023232 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:08.023590 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.024257 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.025601 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.026219 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.026287 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.026296 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.026657 (+ 361us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.026707 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:08:08.026831 4463 raft_consensus.cc:2454] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 0.
I20260807 20:08:08.026893 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.022796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:32947 (local address 127.3.15.131:50631)
0807 20:08:08.023006 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.023016 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.023065 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.023265 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.023266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.023269 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:08.023480 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.023483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.024335 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.024336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.025512 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.025515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.026435 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.026437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.026566 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.026570 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.026650 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.026690 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":138,"threads_started":1}
I20260807 20:08:08.027189 4687 leader_election.cc:302] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3bdc5f4d3d4414be1222ccd261bafe, a4634369066e46e9a5050423dca462f9; no voters:
I20260807 20:08:08.027326 4840 raft_consensus.cc:2781] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:08.027391 4840 raft_consensus.cc:482] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:08.027413 4840 raft_consensus.cc:3037] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.027361 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f1abdb1cc4d0b80a4bdeef4136fc7" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4aa133b5b4a45439ca14871b866ca5f" is_pre_election: true
I20260807 20:08:08.027566 4612 raft_consensus.cc:2454] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 0.
I20260807 20:08:08.028287 4840 raft_consensus.cc:504] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.028426 4840 leader_election.cc:288] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 election: Requested vote from peers a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829), c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947)
I20260807 20:08:08.028669 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f1abdb1cc4d0b80a4bdeef4136fc7" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4aa133b5b4a45439ca14871b866ca5f"
I20260807 20:08:08.028738 4612 raft_consensus.cc:3037] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.029471 4612 raft_consensus.cc:2454] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 1.
I20260807 20:08:08.029613 4463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "404f1abdb1cc4d0b80a4bdeef4136fc7" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4634369066e46e9a5050423dca462f9"
I20260807 20:08:08.029676 4687 leader_election.cc:302] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3bdc5f4d3d4414be1222ccd261bafe, c4aa133b5b4a45439ca14871b866ca5f; no voters:
I20260807 20:08:08.029709 4463 raft_consensus.cc:3037] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.029754 4840 raft_consensus.cc:2781] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:08.029991 4840 raft_consensus.cc:686] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 LEADER]: Becoming Leader. State: Replica: 5a3bdc5f4d3d4414be1222ccd261bafe, State: Running, Role: LEADER
W20260807 20:08:08.030022 4661 tablet.cc:2369] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:08.030161 4840 consensus_queue.cc:237] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.030745 4463 raft_consensus.cc:2454] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 1.
I20260807 20:08:08.031275 4295 catalog_manager.cc:5495] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131). New cstate: current_term: 1 leader_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:08.041889 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.039320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.3.15.131:42157)
0807 20:08:08.039379 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.039381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.039388 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:08.039416 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.039417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.039433 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.039490 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:08.039639 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.040084 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.040887 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.041168 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.041233 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.041528 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.041632 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.041722 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.041776 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.041800 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:08:08.044679 4612 raft_consensus.cc:1261] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Refusing update from remote peer 5a3bdc5f4d3d4414be1222ccd261bafe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:08.044744 4463 raft_consensus.cc:1261] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Refusing update from remote peer 5a3bdc5f4d3d4414be1222ccd261bafe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:08.045112 4842 consensus_queue.cc:1035] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:08.045245 4840 consensus_queue.cc:1035] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:08.047946 4845 mvcc.cc:205] Tried to move back new op lower bound from 7316001947827695616 to 7316001947772362752. Current Snapshot: MvccSnapshot[applied={T|T < 7316001947827695616}]
I20260807 20:08:08.052289 4295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:08.052394 4295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:08.053241 4295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: Sending DeleteTablet for 3 replicas of tablet 404f1abdb1cc4d0b80a4bdeef4136fc7
I20260807 20:08:08.053728 4740 tablet_service.cc:1514] Processing DeleteTablet for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:08 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:39764
I20260807 20:08:08.053802 4590 tablet_service.cc:1514] Processing DeleteTablet for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:08 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:43446
I20260807 20:08:08.054293 4295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:08.054466 4295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:08.055207 4434 tablet_service.cc:1514] Processing DeleteTablet for tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:08 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59622
I20260807 20:08:08.055478 4859 tablet_replica.cc:330] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: stopping tablet replica
I20260807 20:08:08.055585 4859 raft_consensus.cc:2229] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:08.055689 4859 raft_consensus.cc:2258] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:08.055930 4434 tablet_service.cc:1467] Processing CreateTablet for tablet 753b2f80db3643acae90b7016710a13e (DEFAULT_TABLE table=test-table [id=1be59cf430264b4db9157429242db15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:08.056137 4434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753b2f80db3643acae90b7016710a13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:08.056188 4590 tablet_service.cc:1467] Processing CreateTablet for tablet 753b2f80db3643acae90b7016710a13e (DEFAULT_TABLE table=test-table [id=1be59cf430264b4db9157429242db15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:08.056286 4590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753b2f80db3643acae90b7016710a13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:08.056322 4859 ts_tablet_manager.cc:1905] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:08.056978 4740 tablet_service.cc:1467] Processing CreateTablet for tablet 753b2f80db3643acae90b7016710a13e (DEFAULT_TABLE table=test-table [id=1be59cf430264b4db9157429242db15d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:08.057112 4740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 753b2f80db3643acae90b7016710a13e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:08.057785 4832 tablet_bootstrap.cc:492] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Bootstrap starting.
I20260807 20:08:08.058223 4832 tablet_bootstrap.cc:654] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:08.058646 4833 tablet_bootstrap.cc:492] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Bootstrap starting.
I20260807 20:08:08.058720 4859 ts_tablet_manager.cc:1918] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:08.058786 4859 log.cc:1196] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/404f1abdb1cc4d0b80a4bdeef4136fc7
I20260807 20:08:08.059122 4859 ts_tablet_manager.cc:1939] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P a4634369066e46e9a5050423dca462f9: Deleting consensus metadata
I20260807 20:08:08.059129 4833 tablet_bootstrap.cc:654] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:08.059525 4280 catalog_manager.cc:4841] TS a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829): tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 (table test-table [id=d7cd1d4fc3a34cf5b0e767845b7f26a0]) successfully deleted
I20260807 20:08:08.059913 4832 tablet_bootstrap.cc:492] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: No bootstrap required, opened a new log
I20260807 20:08:08.059963 4832 ts_tablet_manager.cc:1397] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 20:08:08.059954 4834 tablet_bootstrap.cc:492] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Bootstrap starting.
I20260807 20:08:08.060143 4832 raft_consensus.cc:348] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.060249 4832 raft_consensus.cc:374] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:08.060295 4832 raft_consensus.cc:729] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4634369066e46e9a5050423dca462f9, State: Initialized, Role: FOLLOWER
I20260807 20:08:08.060343 4832 consensus_queue.cc:260] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.060421 4833 tablet_bootstrap.cc:492] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: No bootstrap required, opened a new log
I20260807 20:08:08.060436 4832 ts_tablet_manager.cc:1428] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:08.060463 4833 ts_tablet_manager.cc:1397] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:08:08.060636 4833 raft_consensus.cc:348] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.060711 4833 raft_consensus.cc:374] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:08.060734 4833 raft_consensus.cc:729] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3bdc5f4d3d4414be1222ccd261bafe, State: Initialized, Role: FOLLOWER
I20260807 20:08:08.060730 4834 tablet_bootstrap.cc:654] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:08.060793 4833 consensus_queue.cc:260] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.060873 4833 ts_tablet_manager.cc:1428] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:08.061718 4857 tablet_replica.cc:330] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: stopping tablet replica
I20260807 20:08:08.061836 4857 raft_consensus.cc:2229] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:08.061879 4834 tablet_bootstrap.cc:492] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: No bootstrap required, opened a new log
I20260807 20:08:08.061944 4834 ts_tablet_manager.cc:1397] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:08:08.061988 4857 raft_consensus.cc:2258] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:08.062095 4834 raft_consensus.cc:348] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.062152 4858 tablet_replica.cc:330] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: stopping tablet replica
I20260807 20:08:08.062177 4834 raft_consensus.cc:374] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:08.062211 4834 raft_consensus.cc:729] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4aa133b5b4a45439ca14871b866ca5f, State: Initialized, Role: FOLLOWER
I20260807 20:08:08.062249 4858 raft_consensus.cc:2229] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:08.062258 4834 consensus_queue.cc:260] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.062358 4834 ts_tablet_manager.cc:1428] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:08.062361 4858 raft_consensus.cc:2258] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:08.062970 4857 ts_tablet_manager.cc:1905] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:08.064607 4857 ts_tablet_manager.cc:1918] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:08.064683 4857 log.cc:1196] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/404f1abdb1cc4d0b80a4bdeef4136fc7
I20260807 20:08:08.065004 4857 ts_tablet_manager.cc:1939] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P c4aa133b5b4a45439ca14871b866ca5f: Deleting consensus metadata
I20260807 20:08:08.065433 4858 ts_tablet_manager.cc:1905] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:08.065472 4280 catalog_manager.cc:4841] TS c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947): tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 (table test-table [id=d7cd1d4fc3a34cf5b0e767845b7f26a0]) successfully deleted
I20260807 20:08:08.067152 4858 ts_tablet_manager.cc:1918] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:08.067250 4858 log.cc:1196] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/404f1abdb1cc4d0b80a4bdeef4136fc7
I20260807 20:08:08.067513 4858 ts_tablet_manager.cc:1939] T 404f1abdb1cc4d0b80a4bdeef4136fc7 P 5a3bdc5f4d3d4414be1222ccd261bafe: Deleting consensus metadata
I20260807 20:08:08.068545 4282 catalog_manager.cc:4841] TS 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131:42157): tablet 404f1abdb1cc4d0b80a4bdeef4136fc7 (table test-table [id=d7cd1d4fc3a34cf5b0e767845b7f26a0]) successfully deleted
W20260807 20:08:08.128190 4511 tablet.cc:2369] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:08:08.192271 4809 tablet.cc:2369] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:08.240262 4840 raft_consensus.cc:482] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:08.240370 4840 raft_consensus.cc:504] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.240549 4840 leader_election.cc:288] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829), c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947)
I20260807 20:08:08.240908 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753b2f80db3643acae90b7016710a13e" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4aa133b5b4a45439ca14871b866ca5f" is_pre_election: true
I20260807 20:08:08.240896 4463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753b2f80db3643acae90b7016710a13e" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4634369066e46e9a5050423dca462f9" is_pre_election: true
I20260807 20:08:08.241024 4612 raft_consensus.cc:2454] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 0.
I20260807 20:08:08.241031 4463 raft_consensus.cc:2454] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 0.
I20260807 20:08:08.241235 4687 leader_election.cc:302] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3bdc5f4d3d4414be1222ccd261bafe, c4aa133b5b4a45439ca14871b866ca5f; no voters:
I20260807 20:08:08.241436 4840 raft_consensus.cc:2781] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:08.241482 4840 raft_consensus.cc:482] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:08.241503 4840 raft_consensus.cc:3037] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.242194 4840 raft_consensus.cc:504] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.242336 4840 leader_election.cc:288] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 election: Requested vote from peers a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829), c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947)
I20260807 20:08:08.242516 4463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753b2f80db3643acae90b7016710a13e" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4634369066e46e9a5050423dca462f9"
I20260807 20:08:08.242614 4463 raft_consensus.cc:3037] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.242556 4612 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "753b2f80db3643acae90b7016710a13e" candidate_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4aa133b5b4a45439ca14871b866ca5f"
I20260807 20:08:08.242656 4612 raft_consensus.cc:3037] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.243269 4463 raft_consensus.cc:2454] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 1.
I20260807 20:08:08.243367 4612 raft_consensus.cc:2454] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a3bdc5f4d3d4414be1222ccd261bafe in term 1.
I20260807 20:08:08.243480 4687 leader_election.cc:302] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3bdc5f4d3d4414be1222ccd261bafe, a4634369066e46e9a5050423dca462f9; no voters:
I20260807 20:08:08.243582 4840 raft_consensus.cc:2781] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:08.243634 4840 raft_consensus.cc:686] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 LEADER]: Becoming Leader. State: Replica: 5a3bdc5f4d3d4414be1222ccd261bafe, State: Running, Role: LEADER
I20260807 20:08:08.243716 4840 consensus_queue.cc:237] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.244302 4295 catalog_manager.cc:5495] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131). New cstate: current_term: 1 leader_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } health_report { overall_health: HEALTHY } } }
Aug 07 20:08:08 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:08.316530 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.309608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36445 (local address 127.0.0.1:41460)
0807 20:08:08.309764 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.309774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.309842 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.309996 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.309996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.310352 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:08.310412 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.310414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.311061 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.311062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.311595 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.311596 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.311901 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.311901 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.311943 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.312450 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:08.312455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:08.313902 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:08.315358 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:08.315365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:08.315367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:08.315564 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:08.315738 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:08.315738 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:08.315739 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:08.315797 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:08.316159 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:08.316167 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:08.316224 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.316326 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.316430 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260807 20:08:08.316579 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.309684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.3.15.190:36445)
0807 20:08:08.309760 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.309763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.309838 (+ 75us) server_negotiation.cc:407] Connection header received
0807 20:08:08.309888 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.309889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.309898 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.309985 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:08.310476 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.310987 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.311661 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.311838 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.311887 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.313985 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:08.313987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:08.313987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:08.313993 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:08.315288 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:08.315629 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:08.315630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:08.315630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:08.315680 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:08.315849 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:08.315850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:08.315850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:08.315992 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:08.316130 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.316364 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.316482 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":50}
I20260807 20:08:08.318704 4295 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41460:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 20:08:08.318997 4295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:08.320520 4740 tablet_service.cc:1467] Processing CreateTablet for tablet 832707dc3ced493b8f65f63196daeeac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb5854cf55444497b6bca12ffd5f05ad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:08.320564 4434 tablet_service.cc:1467] Processing CreateTablet for tablet 832707dc3ced493b8f65f63196daeeac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb5854cf55444497b6bca12ffd5f05ad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:08.320588 4590 tablet_service.cc:1467] Processing CreateTablet for tablet 832707dc3ced493b8f65f63196daeeac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fb5854cf55444497b6bca12ffd5f05ad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:08.320739 4590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 832707dc3ced493b8f65f63196daeeac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:08.320739 4434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 832707dc3ced493b8f65f63196daeeac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:08.320746 4740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 832707dc3ced493b8f65f63196daeeac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:08.322014 4833 tablet_bootstrap.cc:492] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: Bootstrap starting.
I20260807 20:08:08.322285 4832 tablet_bootstrap.cc:492] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: Bootstrap starting.
I20260807 20:08:08.322479 4833 tablet_bootstrap.cc:654] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:08.322642 4832 tablet_bootstrap.cc:654] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:08.322973 4834 tablet_bootstrap.cc:492] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: Bootstrap starting.
I20260807 20:08:08.323102 4833 tablet_bootstrap.cc:492] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: No bootstrap required, opened a new log
I20260807 20:08:08.323155 4833 ts_tablet_manager.cc:1397] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:08.323279 4833 raft_consensus.cc:348] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.323359 4834 tablet_bootstrap.cc:654] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:08.323367 4833 raft_consensus.cc:374] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:08.323405 4833 raft_consensus.cc:729] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a3bdc5f4d3d4414be1222ccd261bafe, State: Initialized, Role: FOLLOWER
I20260807 20:08:08.323449 4833 consensus_queue.cc:260] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.323577 4833 ts_tablet_manager.cc:1428] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:08.323570 4840 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.323650 4840 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.323966 4832 tablet_bootstrap.cc:492] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: No bootstrap required, opened a new log
I20260807 20:08:08.324028 4832 ts_tablet_manager.cc:1397] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 20:08:08.324195 4834 tablet_bootstrap.cc:492] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: No bootstrap required, opened a new log
I20260807 20:08:08.324203 4832 raft_consensus.cc:348] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.324251 4834 ts_tablet_manager.cc:1397] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:08.324286 4832 raft_consensus.cc:374] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:08.324326 4832 raft_consensus.cc:729] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4634369066e46e9a5050423dca462f9, State: Initialized, Role: FOLLOWER
I20260807 20:08:08.324380 4832 consensus_queue.cc:260] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.324414 4834 raft_consensus.cc:348] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.324482 4834 raft_consensus.cc:374] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:08.324496 4832 ts_tablet_manager.cc:1428] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:08.324513 4834 raft_consensus.cc:729] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4aa133b5b4a45439ca14871b866ca5f, State: Initialized, Role: FOLLOWER
I20260807 20:08:08.324565 4834 consensus_queue.cc:260] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.324659 4834 ts_tablet_manager.cc:1428] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:08.324635 4838 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.324704 4838 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.324744 4839 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.324868 4839 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.332027 4839 raft_consensus.cc:482] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:08.332106 4839 raft_consensus.cc:504] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.332448 4839 leader_election.cc:288] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829), 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131:42157)
I20260807 20:08:08.335489 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.332640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:55885 (local address 127.3.15.129:43829)
0807 20:08:08.332686 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.332689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.332758 (+ 69us) server_negotiation.cc:407] Connection header received
0807 20:08:08.332859 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.332860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.332872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.332940 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:08.333167 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.333670 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.334802 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.335139 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.335226 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.335236 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.335384 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.335421 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:08:08.335508 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.332705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:53089 (local address 127.3.15.131:42157)
0807 20:08:08.332748 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.332750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.332952 (+ 202us) server_negotiation.cc:407] Connection header received
0807 20:08:08.332990 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.332991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.333000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.333064 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:08.333211 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.333725 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.334747 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.335139 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.335226 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.335236 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.335390 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.335450 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:08.335539 4873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.332586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42157 (local address 127.3.15.130:53089)
0807 20:08:08.332884 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.332893 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.332944 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.333072 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.333073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.333074 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:08.333126 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.333143 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.333811 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.333812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.334664 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.334667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.335248 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.335249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.335325 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.335332 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.335385 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.335435 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":15,"thread_start_us":56,"threads_started":1}
I20260807 20:08:08.335666 4874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.332586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:43829 (local address 127.3.15.130:55885)
0807 20:08:08.332704 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.332714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.332797 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.332997 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.332999 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.333001 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:08.333094 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.333097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.333730 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.333731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.334667 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.334670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.335247 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.335248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.335322 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.335326 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.335380 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.335419 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260807 20:08:08.335762 4761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832707dc3ced493b8f65f63196daeeac" candidate_uuid: "c4aa133b5b4a45439ca14871b866ca5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" is_pre_election: true
I20260807 20:08:08.335915 4761 raft_consensus.cc:2454] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4aa133b5b4a45439ca14871b866ca5f in term 0.
I20260807 20:08:08.336117 4463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832707dc3ced493b8f65f63196daeeac" candidate_uuid: "c4aa133b5b4a45439ca14871b866ca5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4634369066e46e9a5050423dca462f9" is_pre_election: true
I20260807 20:08:08.336130 4540 leader_election.cc:302] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a3bdc5f4d3d4414be1222ccd261bafe, c4aa133b5b4a45439ca14871b866ca5f; no voters:
I20260807 20:08:08.336205 4463 raft_consensus.cc:2454] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4aa133b5b4a45439ca14871b866ca5f in term 0.
I20260807 20:08:08.336256 4839 raft_consensus.cc:2781] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:08.336294 4839 raft_consensus.cc:482] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:08.336320 4839 raft_consensus.cc:3037] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.337057 4839 raft_consensus.cc:504] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.337198 4839 leader_election.cc:288] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [CANDIDATE]: Term 1 election: Requested vote from peers a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829), 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131:42157)
I20260807 20:08:08.337343 4463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832707dc3ced493b8f65f63196daeeac" candidate_uuid: "c4aa133b5b4a45439ca14871b866ca5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4634369066e46e9a5050423dca462f9"
I20260807 20:08:08.337406 4463 raft_consensus.cc:3037] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.338053 4463 raft_consensus.cc:2454] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4aa133b5b4a45439ca14871b866ca5f in term 1.
I20260807 20:08:08.338145 4761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "832707dc3ced493b8f65f63196daeeac" candidate_uuid: "c4aa133b5b4a45439ca14871b866ca5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe"
I20260807 20:08:08.338260 4761 raft_consensus.cc:3037] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:08.338244 4538 leader_election.cc:302] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4634369066e46e9a5050423dca462f9, c4aa133b5b4a45439ca14871b866ca5f; no voters:
I20260807 20:08:08.338357 4839 raft_consensus.cc:2781] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:08.338500 4839 raft_consensus.cc:686] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [term 1 LEADER]: Becoming Leader. State: Replica: c4aa133b5b4a45439ca14871b866ca5f, State: Running, Role: LEADER
I20260807 20:08:08.338567 4839 consensus_queue.cc:237] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } }
I20260807 20:08:08.339078 4875 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: TxnStatusTablet state changed. Reason: New leader c4aa133b5b4a45439ca14871b866ca5f. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.339145 4761 raft_consensus.cc:2454] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4aa133b5b4a45439ca14871b866ca5f in term 1.
I20260807 20:08:08.339178 4875 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:08.339221 4295 catalog_manager.cc:5495] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f reported cstate change: term changed from 0 to 1, leader changed from <none> to c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130). New cstate: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } health_report { overall_health: UNKNOWN } } }
I20260807 20:08:08.339395 4877 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 20:08:08.339511 4877 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 20:08:08.345873 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.342756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:32947 (local address 127.0.0.1:43448)
0807 20:08:08.342926 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.342938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.342992 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.343160 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.343161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.343162 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:08.343205 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.343207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.344113 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.344115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.344749 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.344752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.345333 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.345334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.345406 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.345422 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.345728 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.345735 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.345770 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.345815 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 20:08:08.345876 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.342824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43448 (local address 127.3.15.130:32947)
0807 20:08:08.342872 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.342874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.342995 (+ 121us) server_negotiation.cc:407] Connection header received
0807 20:08:08.343055 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.343056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.343075 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.343156 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:08.343249 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.344024 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.344901 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.345244 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.345319 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.345524 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.345654 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.345734 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.345784 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.345806 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:08.347291 4463 raft_consensus.cc:1261] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Refusing update from remote peer c4aa133b5b4a45439ca14871b866ca5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:08.347309 4761 raft_consensus.cc:1261] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Refusing update from remote peer c4aa133b5b4a45439ca14871b866ca5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:08.347522 4839 consensus_queue.cc:1035] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:08.347657 4875 consensus_queue.cc:1035] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:08.348259 4838 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: TxnStatusTablet state changed. Reason: New leader c4aa133b5b4a45439ca14871b866ca5f. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.348354 4838 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.348480 4840 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: TxnStatusTablet state changed. Reason: New leader c4aa133b5b4a45439ca14871b866ca5f. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.348577 4840 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.349182 4851 mvcc.cc:205] Tried to move back new op lower bound from 7316001949068242944 to 7316001949035307008. Current Snapshot: MvccSnapshot[applied={T|T < 7316001949068242944}]
I20260807 20:08:08.349313 4875 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.349483 4846 mvcc.cc:205] Tried to move back new op lower bound from 7316001949068242944 to 7316001949035307008. Current Snapshot: MvccSnapshot[applied={T|T < 7316001949068242944}]
I20260807 20:08:08.349619 4875 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:08.349586 4838 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.349668 4838 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P a4634369066e46e9a5050423dca462f9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.351372 4839 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.351470 4839 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P c4aa133b5b4a45439ca14871b866ca5f: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:08.351747 4840 tablet_replica.cc:439] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c4aa133b5b4a45439ca14871b866ca5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 } } peers { permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 } } peers { permanent_uuid: "5a3bdc5f4d3d4414be1222ccd261bafe" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42157 } } }
I20260807 20:08:08.351828 4840 tablet_replica.cc:442] T 832707dc3ced493b8f65f63196daeeac P 5a3bdc5f4d3d4414be1222ccd261bafe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:08.357028 4831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.354322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:38611 (local address 127.3.15.130:32947)
0807 20:08:08.354363 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.354365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.354423 (+ 58us) server_negotiation.cc:407] Connection header received
0807 20:08:08.354495 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.354496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.354507 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.354566 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:08.354656 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.355366 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.356180 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.356507 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.356582 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.356753 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.356822 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.356883 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.356936 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.356956 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:08:08.357038 4814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.354261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:32947 (local address 127.3.15.131:38611)
0807 20:08:08.354364 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.354373 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.354436 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.354569 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.354570 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.354571 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:08.354611 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.354613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.355450 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.355451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.356086 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.356088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.356590 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.356592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.356655 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.356670 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.356885 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.356887 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.356935 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.356980 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 20:08:08.359675 4612 raft_consensus.cc:1261] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Refusing update from remote peer 5a3bdc5f4d3d4414be1222ccd261bafe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:08.359715 4463 raft_consensus.cc:1261] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Refusing update from remote peer 5a3bdc5f4d3d4414be1222ccd261bafe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:08.359908 4842 consensus_queue.cc:1035] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4aa133b5b4a45439ca14871b866ca5f" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 32947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:08.359998 4840 consensus_queue.cc:1035] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4634369066e46e9a5050423dca462f9" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:08.371011 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.367454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:43701 (local address 127.3.15.131:42157)
0807 20:08:08.367496 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.367498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.367765 (+ 267us) server_negotiation.cc:407] Connection header received
0807 20:08:08.367841 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.367842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.367855 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.367907 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:08.368072 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.368740 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.370049 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.370296 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.370349 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.370634 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.370675 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.370724 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.370900 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.370928 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:08:08.371016 4889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.367382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42157 (local address 127.3.15.130:43701)
0807 20:08:08.367594 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:08.367605 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:08.367780 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:08.367910 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:08.367911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:08.367912 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:08.367995 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.367997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.368868 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.368869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.369963 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:08.369967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.370374 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.370375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:08.370465 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.370483 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:08.370727 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:08.370730 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:08.370806 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 20:08:08.370859 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260807 20:08:08.379307 4294 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:08.379441 4294 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:08.380136 4294 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8087b440d6674c198945a9bcfbc0fa96: Sending DeleteTablet for 3 replicas of tablet 753b2f80db3643acae90b7016710a13e
I20260807 20:08:08.380409 4740 tablet_service.cc:1514] Processing DeleteTablet for tablet 753b2f80db3643acae90b7016710a13e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:08 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:39764
I20260807 20:08:08.380425 4434 tablet_service.cc:1514] Processing DeleteTablet for tablet 753b2f80db3643acae90b7016710a13e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:08 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59622
I20260807 20:08:08.380448 4590 tablet_service.cc:1514] Processing DeleteTablet for tablet 753b2f80db3643acae90b7016710a13e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:08 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:43446
I20260807 20:08:08.380532 4857 tablet_replica.cc:330] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: stopping tablet replica
I20260807 20:08:08.380532 4858 tablet_replica.cc:330] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: stopping tablet replica
I20260807 20:08:08.380577 4857 raft_consensus.cc:2229] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:08.380596 4858 raft_consensus.cc:2229] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:08.380609 4857 raft_consensus.cc:2258] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:08.380652 4858 raft_consensus.cc:2258] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:08.380738 4859 tablet_replica.cc:330] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: stopping tablet replica
I20260807 20:08:08.380837 4859 raft_consensus.cc:2229] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:08.380879 4859 raft_consensus.cc:2258] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:08.380971 4857 ts_tablet_manager.cc:1905] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:08.381224 4858 ts_tablet_manager.cc:1905] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:08.382589 4857 ts_tablet_manager.cc:1918] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:08.382650 4857 log.cc:1196] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/753b2f80db3643acae90b7016710a13e
I20260807 20:08:08.382880 4857 ts_tablet_manager.cc:1939] T 753b2f80db3643acae90b7016710a13e P c4aa133b5b4a45439ca14871b866ca5f: Deleting consensus metadata
I20260807 20:08:08.382941 4858 ts_tablet_manager.cc:1918] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:08.382995 4858 log.cc:1196] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/753b2f80db3643acae90b7016710a13e
I20260807 20:08:08.383248 4858 ts_tablet_manager.cc:1939] T 753b2f80db3643acae90b7016710a13e P 5a3bdc5f4d3d4414be1222ccd261bafe: Deleting consensus metadata
I20260807 20:08:08.383266 4280 catalog_manager.cc:4841] TS c4aa133b5b4a45439ca14871b866ca5f (127.3.15.130:32947): tablet 753b2f80db3643acae90b7016710a13e (table test-table [id=1be59cf430264b4db9157429242db15d]) successfully deleted
I20260807 20:08:08.383740 4859 ts_tablet_manager.cc:1905] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:08.383728 4282 catalog_manager.cc:4841] TS 5a3bdc5f4d3d4414be1222ccd261bafe (127.3.15.131:42157): tablet 753b2f80db3643acae90b7016710a13e (table test-table [id=1be59cf430264b4db9157429242db15d]) successfully deleted
I20260807 20:08:08.385401 4859 ts_tablet_manager.cc:1918] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:08.385473 4859 log.cc:1196] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/753b2f80db3643acae90b7016710a13e
I20260807 20:08:08.385805 4859 ts_tablet_manager.cc:1939] T 753b2f80db3643acae90b7016710a13e P a4634369066e46e9a5050423dca462f9: Deleting consensus metadata
I20260807 20:08:08.386162 4280 catalog_manager.cc:4841] TS a4634369066e46e9a5050423dca462f9 (127.3.15.129:43829): tablet 753b2f80db3643acae90b7016710a13e (table test-table [id=1be59cf430264b4db9157429242db15d]) successfully deleted
Aug 07 20:08:08 dist-test-slave-1x4g krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133287, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:08.392571 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.383566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59626 (local address 127.3.15.129:43829)
0807 20:08:08.383623 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.383625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.383893 (+ 268us) server_negotiation.cc:407] Connection header received
0807 20:08:08.384006 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.384007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.384021 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.384071 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:08.384717 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.385264 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.386168 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.386444 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.386482 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.389637 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:08.389648 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:08.389648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:08.389656 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:08.391259 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:08.391693 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:08.391697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:08.391697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:08.391789 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:08.392005 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:08.392006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:08.392006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:08.392162 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:08.392325 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.392389 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.392471 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 20:08:08.393045 4504 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59626
I20260807 20:08:08.400441 4348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:08.394187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41474 (local address 127.3.15.190:36445)
0807 20:08:08.394254 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:08.394256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:08.394266 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:08.394325 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:08.394326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:08.394338 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:08.394422 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:08.394848 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.395625 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.396279 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:08.396600 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:08.396687 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:08.397787 (+ 1100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:08.397789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:08.397789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:08.397797 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:08.399298 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:08.399611 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:08.399612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:08.399612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:08.399676 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:08.399855 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:08.399856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:08.399856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:08.399991 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:08.400103 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:08.400253 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:08:08.400336 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 20:08:08.400700 4294 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41474
I20260807 20:08:08.401340 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 4372
I20260807 20:08:08.410862 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 4520
I20260807 20:08:08.418485 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 4669
I20260807 20:08:08.426278 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 4263
2026-08-07T20:08:08Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2392 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:08 dist-test-slave-1x4g krb5kdc[4897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:08 dist-test-slave-1x4g krb5kdc[4897](info): set up 2 sockets
Aug 07 20:08:08 dist-test-slave-1x4g krb5kdc[4897](info): commencing operation
krb5kdc: starting...
W20260807 20:08:09.539400 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260807 20:08:09.772571 3134 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.233s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260807 20:08:09.972800 3134 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.200s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260807 20:08:10.117551 3134 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.145s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/test-user.keytab.
W20260807 20:08:10.699280 3134 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.582s user 0.001s sys 0.001s
Aug 07 20:08:10 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
W20260807 20:08:10.869287 3134 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.15.190: real 0.146s user 0.002s sys 0.002s
I20260807 20:08:10.869704 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:38459
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:38459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:10.952338 4914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:10.952584 4914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:10.952618 4914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:10.953758 4914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:10.953825 4914 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 20:08:10.953848 4914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:10.953871 4914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:10.953898 4914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:10.953929 4914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:10.955467 4914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:38459
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:38459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:10.955747 4914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:10.955972 4914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:10.960904 4922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:10.960886 4919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:10.960870 4920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:10.961264 4914 server_base.cc:1034] running on GCE node
I20260807 20:08:10.961511 4914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:10.961788 4914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:10.962949 4914 hybrid_clock.cc:648] HybridClock initialized: now 1786133290962938 us; error 42 us; skew 500 ppm
Aug 07 20:08:10 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133290, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:10.969692 4914 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:10.970247 4914 webserver.cc:460] Webserver started at http://127.3.15.190:38177/ using document root <none> and password file <none>
I20260807 20:08:10.970489 4914 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:10.970556 4914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:10.970708 4914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:10.971889 4914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "7649da9749194a5494794274f6176533"
format_stamp: "Formatted at 2026-08-07 20:08:10 on dist-test-slave-1x4g"
server_key: "247f097658a3289680efe91695ab052e"
server_key_iv: "b36fd8e2c07baf5ef852d91bd7a825e8"
server_key_version: "encryptionkey@0"
I20260807 20:08:10.972275 4914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "7649da9749194a5494794274f6176533"
format_stamp: "Formatted at 2026-08-07 20:08:10 on dist-test-slave-1x4g"
server_key: "247f097658a3289680efe91695ab052e"
server_key_iv: "b36fd8e2c07baf5ef852d91bd7a825e8"
server_key_version: "encryptionkey@0"
I20260807 20:08:10.973760 4914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:10.974603 4930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:10.974824 4914 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:10.975000 4914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "7649da9749194a5494794274f6176533"
format_stamp: "Formatted at 2026-08-07 20:08:10 on dist-test-slave-1x4g"
server_key: "247f097658a3289680efe91695ab052e"
server_key_iv: "b36fd8e2c07baf5ef852d91bd7a825e8"
server_key_version: "encryptionkey@0"
I20260807 20:08:10.975090 4914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:10.997224 4914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:10.999243 4914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:10.999408 4914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:11.004362 4914 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:38459
I20260807 20:08:11.004413 4993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:38459 every 8 connection(s)
I20260807 20:08:11.004750 4914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:11.005434 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 4914
I20260807 20:08:11.005517 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:11.005470 4994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.005697 3134 external_mini_cluster.cc:1428] Setting key 0e55235c728902bcaac5c33cbf812f04
I20260807 20:08:11.008986 4994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Bootstrap starting.
I20260807 20:08:11.009706 4994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.010004 4994 log.cc:824] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:11.010766 4994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: No bootstrap required, opened a new log
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:11.012923 4994 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7649da9749194a5494794274f6176533" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38459 } }
I20260807 20:08:11.013108 4994 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.013146 4994 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7649da9749194a5494794274f6176533, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.013293 4994 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7649da9749194a5494794274f6176533" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38459 } }
I20260807 20:08:11.013381 4994 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:11.013435 4994 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:11.013500 4994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:11.014220 4994 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7649da9749194a5494794274f6176533" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38459 } }
I20260807 20:08:11.014312 4994 leader_election.cc:302] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7649da9749194a5494794274f6176533; no voters:
I20260807 20:08:11.014534 4994 leader_election.cc:288] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:11.014566 4999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:11.014678 4999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [term 1 LEADER]: Becoming Leader. State: Replica: 7649da9749194a5494794274f6176533, State: Running, Role: LEADER
I20260807 20:08:11.014762 4999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7649da9749194a5494794274f6176533" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38459 } }
I20260807 20:08:11.014797 4994 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:11.015110 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7649da9749194a5494794274f6176533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7649da9749194a5494794274f6176533" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38459 } } }
I20260807 20:08:11.015288 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:11.015489 4999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7649da9749194a5494794274f6176533. Latest consensus state: current_term: 1 leader_uuid: "7649da9749194a5494794274f6176533" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7649da9749194a5494794274f6176533" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38459 } } }
I20260807 20:08:11.015563 4999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:11.015978 5009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:11.016593 5009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:11.017460 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.006330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36030 (local address 127.3.15.190:38459)
0807 20:08:11.006535 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.006545 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.006579 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:08:11.007114 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.007123 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.007192 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.007350 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.007954 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.009393 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.010212 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.010740 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.010850 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.013536 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.013541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.013541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.013554 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.015664 (+ 2110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.016103 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.016106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.016106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.016224 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.016458 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.016459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.016460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.016823 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.016984 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.017114 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.017256 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260807 20:08:11.018575 5009 catalog_manager.cc:1347] Generated new cluster ID: aa2fff232ae1435d893ba7d5e5181cd3
I20260807 20:08:11.018627 5009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:11.033910 5009 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:11.034497 5009 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:11.037043 5009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Generated new TSK 0
I20260807 20:08:11.037230 5009 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:11.071657 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38459
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:11.156715 5022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:11.156953 5022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:11.156980 5022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:11.158241 5022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:11.158281 5022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:11.158303 5022 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 20:08:11.158315 5022 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:11.158332 5022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:11.159740 5022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:38459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:11.159941 5022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:11.160115 5022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:11.165088 5028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:11.165097 5029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:11.165272 5022 server_base.cc:1034] running on GCE node
W20260807 20:08:11.165109 5031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:11.165593 5022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:11.165843 5022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:11.167002 5022 hybrid_clock.cc:648] HybridClock initialized: now 1786133291166986 us; error 29 us; skew 500 ppm
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:11.169895 5022 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:11.170383 5022 webserver.cc:460] Webserver started at http://127.3.15.129:33769/ using document root <none> and password file <none>
I20260807 20:08:11.170634 5022 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:11.170702 5022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:11.170857 5022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:11.171923 5022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "ed1835c891601ab0099fd512de7235f0"
server_key_iv: "41b160968f12218c1cff29114b682b9c"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.172299 5022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "ed1835c891601ab0099fd512de7235f0"
server_key_iv: "41b160968f12218c1cff29114b682b9c"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.173715 5022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:08:11.174443 5038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.174609 5022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:11.174719 5022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "ed1835c891601ab0099fd512de7235f0"
server_key_iv: "41b160968f12218c1cff29114b682b9c"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.174794 5022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:11.182196 5022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:11.184514 5022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:11.184677 5022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:11.185257 5022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:11.185317 5022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.185361 5022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:11.185389 5022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:11.192791 5022 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:46013
I20260807 20:08:11.192914 5160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:46013 every 8 connection(s)
I20260807 20:08:11.193192 5022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:11.195648 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.185682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:42115 (local address 127.3.15.190:38459)
0807 20:08:11.185753 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.185755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.185972 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:08:11.186440 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.186441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.186457 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.186538 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.187298 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.188146 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.189893 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.190248 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.190327 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.192952 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.192954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.192954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.192962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.194507 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.194874 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.194875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.194875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.194927 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.195113 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.195113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.195114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.195220 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.195271 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.195510 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.195583 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:08:11.195808 5048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.185657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.3.15.129:42115)
0807 20:08:11.185831 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.185871 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.186346 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.186622 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.186626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.186984 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.187195 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.187201 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.188276 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.188277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.189778 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.189782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.190338 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.190340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.190506 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.191011 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.191023 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.192809 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.194585 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.194591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.194593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.194805 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.194969 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.194970 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.194970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.195053 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.195278 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.195283 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.195394 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.195482 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.195599 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":102,"threads_started":1}
I20260807 20:08:11.196305 5161 heartbeater.cc:347] Connected to a master server at 127.3.15.190:38459
I20260807 20:08:11.196390 5161 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:11.196583 5161 heartbeater.cc:511] Master 127.3.15.190:38459 requested a full tablet report, sending...
I20260807 20:08:11.197198 4947 ts_manager.cc:194] Registered new tserver with Master: 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013)
I20260807 20:08:11.197211 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 5022
I20260807 20:08:11.197310 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:11.197512 3134 external_mini_cluster.cc:1428] Setting key c7321fe2bb4a309a23b5ff38f4581fda
I20260807 20:08:11.198151 4947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:42115
I20260807 20:08:11.203434 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.197562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:43719 (local address 127.3.15.190:38459)
0807 20:08:11.197603 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.197605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.197753 (+ 148us) server_negotiation.cc:407] Connection header received
0807 20:08:11.197855 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.197856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.197866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.197911 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.198315 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.199033 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.199717 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.199921 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.200002 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.200969 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.200970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.200971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.200978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.202320 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.202638 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.202639 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.202639 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.202694 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.202905 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.202906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.202906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.203023 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.203094 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.203274 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.203343 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:08:11.203464 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.197495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.3.15.129:43719)
0807 20:08:11.197693 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.197704 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.197779 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.197924 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.197925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.198149 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.198218 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.198220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.199114 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.199115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.199618 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.199619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.200001 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.200002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.200079 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.200514 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.200519 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.200875 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.202385 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.202386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.202387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.202563 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.202772 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.202773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.202774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.202834 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.203098 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.203100 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.203144 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.203234 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.203342 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":139,"threads_started":1}
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:11.224102 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38459
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:11.306726 5170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:11.306924 5170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:11.306948 5170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:11.307996 5170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:11.308028 5170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:11.308048 5170 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 20:08:11.308061 5170 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:11.308077 5170 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:11.309465 5170 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:38459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:11.309650 5170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:11.309814 5170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:11.314766 5176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:11.314822 5177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:11.314764 5179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:11.314844 5170 server_base.cc:1034] running on GCE node
I20260807 20:08:11.315346 5170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:11.315632 5170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:11.316831 5170 hybrid_clock.cc:648] HybridClock initialized: now 1786133291316809 us; error 38 us; skew 500 ppm
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:11.319298 5170 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:11.319741 5170 webserver.cc:460] Webserver started at http://127.3.15.130:45135/ using document root <none> and password file <none>
I20260807 20:08:11.319973 5170 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:11.320040 5170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:11.320178 5170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:11.321345 5170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "90edb56383614ba18f33dca3c81243f3"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "143639b1b0c0b9716c77eb0bf58034e8"
server_key_iv: "e10b59253b48d94d6589d05c665a317d"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.321717 5170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "90edb56383614ba18f33dca3c81243f3"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "143639b1b0c0b9716c77eb0bf58034e8"
server_key_iv: "e10b59253b48d94d6589d05c665a317d"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.323057 5170 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:11.323832 5187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.324026 5170 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.324122 5170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "90edb56383614ba18f33dca3c81243f3"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "143639b1b0c0b9716c77eb0bf58034e8"
server_key_iv: "e10b59253b48d94d6589d05c665a317d"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.324178 5170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:11.333427 5170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:11.335626 5170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:11.335738 5170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:11.336184 5170 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:11.336221 5170 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.336242 5170 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:11.336261 5170 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.343366 5170 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:34127
I20260807 20:08:11.343453 5308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:34127 every 8 connection(s)
I20260807 20:08:11.343756 5170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:11.347471 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.336827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:35367 (local address 127.3.15.190:38459)
0807 20:08:11.336880 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.336887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.337139 (+ 252us) server_negotiation.cc:407] Connection header received
0807 20:08:11.337773 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.337776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.337791 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.337883 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.338838 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.339844 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.341544 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.341782 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.341871 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.344673 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.344675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.344675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.344691 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.346258 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.346621 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.346622 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.346622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.346676 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.346932 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.346932 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.346933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.347047 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.347106 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.347334 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.347404 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:11.347652 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.336818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.3.15.130:35367)
0807 20:08:11.337007 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.337042 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.337645 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.338002 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.338009 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.338416 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.338732 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.338740 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.339963 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.339964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.341447 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.341450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.341881 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.341882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.342085 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.342722 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.342732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.344572 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.346331 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.346337 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.346339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.346550 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.346736 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.346737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.346737 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.346839 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.347114 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.347118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.347228 (+ 110us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.347309 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.347429 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260807 20:08:11.348143 5309 heartbeater.cc:347] Connected to a master server at 127.3.15.190:38459
I20260807 20:08:11.348235 5309 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:11.348453 5309 heartbeater.cc:511] Master 127.3.15.190:38459 requested a full tablet report, sending...
I20260807 20:08:11.348960 4947 ts_manager.cc:194] Registered new tserver with Master: 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:11.349393 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 5170
I20260807 20:08:11.349516 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:11.349498 4947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:35367
I20260807 20:08:11.349737 3134 external_mini_cluster.cc:1428] Setting key 3e1c139b9aea935b465dc121dfaa1ec2
I20260807 20:08:11.355357 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.349287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:47715 (local address 127.3.15.190:38459)
0807 20:08:11.349329 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.349331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.349475 (+ 144us) server_negotiation.cc:407] Connection header received
0807 20:08:11.349568 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.349570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.349581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.349613 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.350176 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.350645 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.351454 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.351681 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.351758 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.352756 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.352757 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.352757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.352769 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.354190 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.354614 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.354614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.354615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.354663 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.354867 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.354867 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.354868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.355000 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.355051 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.355246 (+ 195us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.355302 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:11.355543 5315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.349228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.3.15.130:47715)
0807 20:08:11.349386 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.349400 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.349483 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.349761 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.349761 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.350026 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.350089 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.350091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.350727 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.350728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.351359 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.351362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.351772 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.351773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.351867 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.352283 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.352288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.352659 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.354270 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.354272 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.354272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.354533 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.354725 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.354726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.354726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.354804 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.355072 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.355073 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.355131 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.355233 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.355368 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:11.380100 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38459
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:11.465492 5318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:11.465764 5318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:11.465809 5318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:11.467015 5318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:11.467095 5318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:11.467128 5318 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 20:08:11.467154 5318 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:11.467196 5318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:11.468704 5318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:38459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:11.469030 5318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:11.469287 5318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:11.474566 5324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:11.474567 5323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:11.474642 5327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:11.474738 5318 server_base.cc:1034] running on GCE node
I20260807 20:08:11.474961 5318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:11.475245 5318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:11.476413 5318 hybrid_clock.cc:648] HybridClock initialized: now 1786133291476392 us; error 37 us; skew 500 ppm
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:11.479206 5318 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:11.479735 5318 webserver.cc:460] Webserver started at http://127.3.15.131:44899/ using document root <none> and password file <none>
I20260807 20:08:11.479950 5318 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:11.479991 5318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:11.480093 5318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:11.481299 5318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "2883d7ebb37546619fd83e85f9c2ca14"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "fe08de41114f08f667630b5f2125958f"
server_key_iv: "6916475b8f08f325071eeefe3ff5b103"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.481621 5318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "2883d7ebb37546619fd83e85f9c2ca14"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "fe08de41114f08f667630b5f2125958f"
server_key_iv: "6916475b8f08f325071eeefe3ff5b103"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.482992 5318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:11.483687 5335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.483883 5318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:11.483974 5318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "2883d7ebb37546619fd83e85f9c2ca14"
format_stamp: "Formatted at 2026-08-07 20:08:11 on dist-test-slave-1x4g"
server_key: "fe08de41114f08f667630b5f2125958f"
server_key_iv: "6916475b8f08f325071eeefe3ff5b103"
server_key_version: "encryptionkey@0"
I20260807 20:08:11.484032 5318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:11.491988 5318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:11.494328 5318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:11.494446 5318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:11.494948 5318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:11.494987 5318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.495009 5318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:11.495023 5318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.502439 5318 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:42239
I20260807 20:08:11.502492 5453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:42239 every 8 connection(s)
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:11.502902 5318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:11.505813 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 5318
I20260807 20:08:11.505957 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:11.506109 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.495617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:56515 (local address 127.3.15.190:38459)
0807 20:08:11.495683 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.495686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.495874 (+ 188us) server_negotiation.cc:407] Connection header received
0807 20:08:11.496502 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.496503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.496517 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.496574 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.497676 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.498543 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.500261 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.500455 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.500508 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.503495 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.503496 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.503497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.503505 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.504720 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.505164 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.505165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.505165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.505222 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.505464 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.505464 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.505465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.505575 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.505629 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.505911 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.506006 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:08:11.506215 3134 external_mini_cluster.cc:1428] Setting key d422f46b3b6522dc4d4921750b0fbfa5
I20260807 20:08:11.506244 5344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.495596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.3.15.131:56515)
0807 20:08:11.495754 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.495794 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.496362 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.496709 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.496716 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.497212 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.497551 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.497559 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.498654 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.498656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.500148 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.500152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.500546 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.500547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.500729 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.501386 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.501397 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.503403 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.504844 (+ 1441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.504851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.504853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.505094 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.505283 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.505283 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.505284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.505385 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.505630 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.505635 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.505753 (+ 118us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.505859 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.506024 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260807 20:08:11.507875 5454 heartbeater.cc:347] Connected to a master server at 127.3.15.190:38459
I20260807 20:08:11.507944 5454 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:11.508133 5454 heartbeater.cc:511] Master 127.3.15.190:38459 requested a full tablet report, sending...
I20260807 20:08:11.508517 4947 ts_manager.cc:194] Registered new tserver with Master: 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239)
I20260807 20:08:11.509050 4947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:56515
I20260807 20:08:11.514243 5460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.508034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.3.15.131:57773)
0807 20:08:11.508170 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.508183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.508265 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.508432 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.508433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.508641 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.508705 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.508707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.509589 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.509590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.510267 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.510269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.510678 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.510679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.510745 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.511198 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.511203 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.511502 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.513157 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.513158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.513158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.513336 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.513520 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.513521 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.513521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.513581 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.513984 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.513985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.514029 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.514092 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.514175 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":52,"thread_start_us":87,"threads_started":1}
I20260807 20:08:11.514256 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.508031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:57773 (local address 127.3.15.190:38459)
0807 20:08:11.508078 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.508080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.508240 (+ 160us) server_negotiation.cc:407] Connection header received
0807 20:08:11.508341 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.508342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.508354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.508413 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.508819 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.509492 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.510345 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.510577 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.510668 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.511582 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.511584 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.511584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.511590 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.513017 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.513403 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.513403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.513404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.513457 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.513678 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.513680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.513680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.513833 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.513925 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.514114 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.514190 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:08:11.518226 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133291, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:11.543968 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.536759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36042 (local address 127.3.15.190:38459)
0807 20:08:11.536873 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.536875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.536885 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:11.536930 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.536930 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.536977 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.537046 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.537667 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.538204 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.538918 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.539120 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.539178 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.541476 (+ 2298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.541478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.541479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.541493 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.542718 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.543118 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.543119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.543120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.543183 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.543390 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.543391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.543391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.543520 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.543627 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.543779 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.543860 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 20:08:11.545886 4947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:11.547657 4947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:11.554994 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.550666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.3.15.130:34127)
0807 20:08:11.550944 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.550949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.550961 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:08:11.551091 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.551114 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.551131 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.551311 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:11.551380 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.552293 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.553680 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.554538 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.554624 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.554694 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.554835 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.554876 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260807 20:08:11.554975 5471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.550471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:34127 (local address 127.0.0.1:49126)
0807 20:08:11.550661 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.550703 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.550964 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.551238 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.551241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.551243 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:11.551318 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.551320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.552373 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.552374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.553564 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.553567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.554625 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.554626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.554714 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.554729 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.554811 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.554873 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":81,"threads_started":1}
I20260807 20:08:11.555697 5472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.550716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.0.0.1:59312)
0807 20:08:11.551012 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.551022 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.551066 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.551796 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.551797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.551799 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:11.551846 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.551847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.553196 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.553197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.554423 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.554427 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.555427 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.555428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.555515 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.555520 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.555578 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.555625 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":60,"threads_started":1}
I20260807 20:08:11.555840 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.550905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59312 (local address 127.3.15.131:42239)
0807 20:08:11.551616 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.551622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.551635 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:08:11.551677 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.551679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.551695 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.551784 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:11.551910 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.553100 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.554536 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.555327 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.555406 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.555464 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.555666 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.555715 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":121,"threads_started":1}
I20260807 20:08:11.555977 5238 tablet_service.cc:1467] Processing CreateTablet for tablet a1f4ef19f40648a28af0e77145a808ac (DEFAULT_TABLE table=test-table [id=a5a6f36273374a5ab59482bc2b1b9267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:11.556412 5238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f4ef19f40648a28af0e77145a808ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.556733 5386 tablet_service.cc:1467] Processing CreateTablet for tablet a1f4ef19f40648a28af0e77145a808ac (DEFAULT_TABLE table=test-table [id=a5a6f36273374a5ab59482bc2b1b9267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:11.556900 5475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.550516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46013 (local address 127.0.0.1:46804)
0807 20:08:11.550926 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.550938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.551045 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.551416 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.551417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.551418 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:11.551465 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.551466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.552967 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.552968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.553963 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.553965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.554794 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.554795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.554865 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.554869 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.554917 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.554957 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":87,"threads_started":1}
I20260807 20:08:11.557128 5386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f4ef19f40648a28af0e77145a808ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.557168 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.550656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46804 (local address 127.3.15.129:46013)
0807 20:08:11.551256 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.551260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.551270 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:11.551313 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.551315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.551334 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.551422 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:11.551563 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.552862 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.554051 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.554629 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.554700 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.554756 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.555013 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.555057 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":93,"threads_started":1}
I20260807 20:08:11.557825 5092 tablet_service.cc:1467] Processing CreateTablet for tablet a1f4ef19f40648a28af0e77145a808ac (DEFAULT_TABLE table=test-table [id=a5a6f36273374a5ab59482bc2b1b9267]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:11.558218 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1f4ef19f40648a28af0e77145a808ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.560487 5477 tablet_bootstrap.cc:492] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Bootstrap starting.
I20260807 20:08:11.560949 5478 tablet_bootstrap.cc:492] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Bootstrap starting.
I20260807 20:08:11.561280 5477 tablet_bootstrap.cc:654] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.561424 5479 tablet_bootstrap.cc:492] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Bootstrap starting.
I20260807 20:08:11.561831 5477 log.cc:824] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:11.561925 5478 tablet_bootstrap.cc:654] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.562112 5479 tablet_bootstrap.cc:654] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.562326 5478 log.cc:824] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:11.562490 5479 log.cc:824] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:11.562736 5477 tablet_bootstrap.cc:492] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: No bootstrap required, opened a new log
I20260807 20:08:11.562822 5477 ts_tablet_manager.cc:1397] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:08:11.563177 5478 tablet_bootstrap.cc:492] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: No bootstrap required, opened a new log
I20260807 20:08:11.563261 5478 ts_tablet_manager.cc:1397] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:08:11.563498 5479 tablet_bootstrap.cc:492] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: No bootstrap required, opened a new log
I20260807 20:08:11.563616 5479 ts_tablet_manager.cc:1397] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:11.565035 5478 raft_consensus.cc:348] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.565194 5478 raft_consensus.cc:374] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.565256 5478 raft_consensus.cc:729] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.565207 5477 raft_consensus.cc:348] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.565425 5477 raft_consensus.cc:374] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.565384 5478 consensus_queue.cc:260] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.565490 5477 raft_consensus.cc:729] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90edb56383614ba18f33dca3c81243f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.565665 5477 consensus_queue.cc:260] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.565702 5454 heartbeater.cc:503] Master 127.3.15.190:38459 was elected leader, sending a full tablet report...
I20260807 20:08:11.565709 5478 ts_tablet_manager.cc:1428] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:08:11.565861 5479 raft_consensus.cc:348] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.565999 5477 ts_tablet_manager.cc:1428] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 20:08:11.566020 5309 heartbeater.cc:503] Master 127.3.15.190:38459 was elected leader, sending a full tablet report...
I20260807 20:08:11.566068 5479 raft_consensus.cc:374] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.566112 5479 raft_consensus.cc:729] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3f0ae4a45d414cbcdfd396dc03c16b, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.566246 5479 consensus_queue.cc:260] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.566532 5479 ts_tablet_manager.cc:1428] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:08:11.566565 5161 heartbeater.cc:503] Master 127.3.15.190:38459 was elected leader, sending a full tablet report...
I20260807 20:08:11.591382 5485 raft_consensus.cc:482] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:11.591501 5485 raft_consensus.cc:504] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.591908 5485 leader_election.cc:288] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239)
W20260807 20:08:11.594308 5310 tablet.cc:2369] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:11.594983 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.592138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46013 (local address 127.3.15.130:37479)
0807 20:08:11.592201 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.592215 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.592290 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.592442 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.592443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.592445 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:11.592540 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.592542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.593286 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.593287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.594090 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.594093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.594745 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.594746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.594832 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.594837 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.594890 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.594933 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:08:11.595043 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.592249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:37479 (local address 127.3.15.129:46013)
0807 20:08:11.592300 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.592303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.592338 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:08:11.592369 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.592370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.592381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.592461 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:11.592641 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.593211 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.594182 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.594663 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.594725 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.594732 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.594910 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.594954 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:08:11.595160 5487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.592257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.3.15.130:38103)
0807 20:08:11.592462 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.592473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.592535 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.592662 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.592663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.592664 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:11.592739 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.592740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.593451 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.593452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.594177 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.594179 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.594845 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.594846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.594959 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.594963 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.595022 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.595059 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":1,"thread_start_us":141,"threads_started":1}
I20260807 20:08:11.595188 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.592370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:38103 (local address 127.3.15.131:42239)
0807 20:08:11.592413 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.592415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.592533 (+ 118us) server_negotiation.cc:407] Connection header received
0807 20:08:11.592578 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.592579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.592593 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.592647 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:11.592848 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.593367 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.594268 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.594760 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.594834 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.594843 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.595044 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.595106 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:11.595322 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f4ef19f40648a28af0e77145a808ac" candidate_uuid: "90edb56383614ba18f33dca3c81243f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" is_pre_election: true
I20260807 20:08:11.595497 5113 raft_consensus.cc:2454] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90edb56383614ba18f33dca3c81243f3 in term 0.
I20260807 20:08:11.595667 5406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f4ef19f40648a28af0e77145a808ac" candidate_uuid: "90edb56383614ba18f33dca3c81243f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2883d7ebb37546619fd83e85f9c2ca14" is_pre_election: true
I20260807 20:08:11.595732 5190 leader_election.cc:302] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3f0ae4a45d414cbcdfd396dc03c16b, 90edb56383614ba18f33dca3c81243f3; no voters:
I20260807 20:08:11.595839 5406 raft_consensus.cc:2454] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90edb56383614ba18f33dca3c81243f3 in term 0.
I20260807 20:08:11.595876 5485 raft_consensus.cc:2781] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:11.595925 5485 raft_consensus.cc:482] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:11.595952 5485 raft_consensus.cc:3037] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:11.597002 5485 raft_consensus.cc:504] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.597105 5485 leader_election.cc:288] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239)
I20260807 20:08:11.597241 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f4ef19f40648a28af0e77145a808ac" candidate_uuid: "90edb56383614ba18f33dca3c81243f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
I20260807 20:08:11.597271 5406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1f4ef19f40648a28af0e77145a808ac" candidate_uuid: "90edb56383614ba18f33dca3c81243f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2883d7ebb37546619fd83e85f9c2ca14"
I20260807 20:08:11.597318 5113 raft_consensus.cc:3037] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:11.597348 5406 raft_consensus.cc:3037] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:11.598253 5113 raft_consensus.cc:2454] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90edb56383614ba18f33dca3c81243f3 in term 1.
I20260807 20:08:11.598333 5406 raft_consensus.cc:2454] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90edb56383614ba18f33dca3c81243f3 in term 1.
I20260807 20:08:11.598416 5190 leader_election.cc:302] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3f0ae4a45d414cbcdfd396dc03c16b, 90edb56383614ba18f33dca3c81243f3; no voters:
I20260807 20:08:11.598508 5485 raft_consensus.cc:2781] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:11.598845 5485 raft_consensus.cc:686] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 1 LEADER]: Becoming Leader. State: Replica: 90edb56383614ba18f33dca3c81243f3, State: Running, Role: LEADER
I20260807 20:08:11.598953 5485 consensus_queue.cc:237] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.599792 4948 catalog_manager.cc:5495] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 90edb56383614ba18f33dca3c81243f3 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "90edb56383614ba18f33dca3c81243f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } health_report { overall_health: UNKNOWN } } }
Aug 07 20:08:11 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133290, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:11.633945 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.625602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36046 (local address 127.3.15.190:38459)
0807 20:08:11.625685 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:11.625688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:11.625935 (+ 247us) server_negotiation.cc:407] Connection header received
0807 20:08:11.626032 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:11.626033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:11.626044 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:11.626118 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:11.626648 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.627209 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.628055 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.628332 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.628415 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.631565 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:11.631567 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:11.631567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:11.631576 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:11.632910 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.633299 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.633300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.633300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.633354 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:11.633530 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:11.633530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:11.633531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:11.633650 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:11.633700 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:11.633806 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 20:08:11.633877 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:08:11.634158 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:11.625523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38459 (local address 127.0.0.1:36046)
0807 20:08:11.625741 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:11.625751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:11.625960 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:11.626137 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:11.626138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:11.626506 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:11.626555 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.626558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.627304 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.627305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.627867 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:11.627869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:11.628416 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:11.628417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:11.628536 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:11.629383 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:11.629389 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:11.631416 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:11.632989 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.632997 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.632999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.633223 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.633397 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:11.633397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:11.633398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:11.633479 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:11.633691 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:11.633697 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:11.633739 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 20:08:11.633783 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:08:11.633932 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":48,"thread_start_us":137,"threads_started":1}
I20260807 20:08:11.636700 4948 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:36046:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 20:08:11.636967 4948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:11.638530 5386 tablet_service.cc:1467] Processing CreateTablet for tablet a63b14addb7749428ea0e17c792e4d0f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36997760fb4c4d8a8a079142727bdfeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:11.638504 5238 tablet_service.cc:1467] Processing CreateTablet for tablet a63b14addb7749428ea0e17c792e4d0f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36997760fb4c4d8a8a079142727bdfeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:11.638549 5092 tablet_service.cc:1467] Processing CreateTablet for tablet a63b14addb7749428ea0e17c792e4d0f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=36997760fb4c4d8a8a079142727bdfeb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:11.638731 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63b14addb7749428ea0e17c792e4d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.638727 5238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63b14addb7749428ea0e17c792e4d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.638751 5386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a63b14addb7749428ea0e17c792e4d0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:11.639995 5479 tablet_bootstrap.cc:492] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: Bootstrap starting.
I20260807 20:08:11.640285 5478 tablet_bootstrap.cc:492] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: Bootstrap starting.
I20260807 20:08:11.640697 5479 tablet_bootstrap.cc:654] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.640854 5478 tablet_bootstrap.cc:654] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.641273 5477 tablet_bootstrap.cc:492] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: Bootstrap starting.
I20260807 20:08:11.641584 5479 tablet_bootstrap.cc:492] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: No bootstrap required, opened a new log
I20260807 20:08:11.641639 5479 ts_tablet_manager.cc:1397] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:11.641744 5478 tablet_bootstrap.cc:492] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: No bootstrap required, opened a new log
I20260807 20:08:11.641746 5477 tablet_bootstrap.cc:654] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:11.641791 5478 ts_tablet_manager.cc:1397] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:11.641765 5479 raft_consensus.cc:348] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.641819 5479 raft_consensus.cc:374] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.641835 5479 raft_consensus.cc:729] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3f0ae4a45d414cbcdfd396dc03c16b, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.641908 5479 consensus_queue.cc:260] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.641948 5478 raft_consensus.cc:348] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.642024 5478 raft_consensus.cc:374] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.642032 5479 ts_tablet_manager.cc:1428] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.642051 5478 raft_consensus.cc:729] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.642123 5478 consensus_queue.cc:260] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.642105 5486 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:11.642220 5478 ts_tablet_manager.cc:1428] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.642248 5486 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:11.642310 5484 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:11.642421 5484 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:11.643091 5477 tablet_bootstrap.cc:492] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: No bootstrap required, opened a new log
I20260807 20:08:11.643163 5477 ts_tablet_manager.cc:1397] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260807 20:08:11.643322 5477 raft_consensus.cc:348] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.643406 5477 raft_consensus.cc:374] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:11.643450 5477 raft_consensus.cc:729] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90edb56383614ba18f33dca3c81243f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:11.643515 5477 consensus_queue.cc:260] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:11.643623 5477 ts_tablet_manager.cc:1428] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:11.643715 5485 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:11.643826 5485 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 20:08:11.693969 5162 tablet.cc:2369] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:08:11.753531 5455 tablet.cc:2369] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:12.020498 5484 raft_consensus.cc:482] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:12.020612 5484 raft_consensus.cc:504] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.021119 5484 leader_election.cc:288] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:12.024127 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.021393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:44817 (local address 127.3.15.130:34127)
0807 20:08:12.021441 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.021443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.021525 (+ 82us) server_negotiation.cc:407] Connection header received
0807 20:08:12.021566 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.021567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.021581 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.021648 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:12.021803 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.022321 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.023399 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.023746 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.023839 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.023847 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.024021 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.024064 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:12.024130 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.021308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46013 (local address 127.3.15.131:57909)
0807 20:08:12.021469 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:12.021486 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:12.021572 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:12.021738 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:12.021739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:12.021740 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:12.021828 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.021829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.022472 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.022472 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.023274 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.023276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.023825 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.023826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.023908 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.023912 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:12.023964 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:08:12.024028 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":2,"thread_start_us":63,"threads_started":1}
I20260807 20:08:12.024140 5473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.021369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:57909 (local address 127.3.15.129:46013)
0807 20:08:12.021422 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.021425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.021578 (+ 153us) server_negotiation.cc:407] Connection header received
0807 20:08:12.021657 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.021659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.021670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.021722 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:12.021899 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.022379 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.023369 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.023739 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.023828 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.023838 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.023988 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.024037 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:08:12.024142 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.021299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:34127 (local address 127.3.15.131:44817)
0807 20:08:12.021435 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:12.021447 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:12.021516 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:12.021659 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:12.021660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:12.021662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:12.021729 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.021731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.022404 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.022405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.023275 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.023278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.023854 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.023855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.023928 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.023932 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:12.024006 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:08:12.024047 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":47,"thread_start_us":78,"threads_started":1}
I20260807 20:08:12.024451 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63b14addb7749428ea0e17c792e4d0f" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" is_pre_election: true
I20260807 20:08:12.024573 5113 raft_consensus.cc:2454] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 0.
I20260807 20:08:12.024689 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63b14addb7749428ea0e17c792e4d0f" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3" is_pre_election: true
I20260807 20:08:12.024827 5260 raft_consensus.cc:2454] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 0.
I20260807 20:08:12.024852 5338 leader_election.cc:302] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:12.025029 5484 raft_consensus.cc:2781] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:12.025081 5484 raft_consensus.cc:482] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:12.025096 5484 raft_consensus.cc:3037] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:12.025842 5484 raft_consensus.cc:504] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.026003 5484 leader_election.cc:288] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:12.026140 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63b14addb7749428ea0e17c792e4d0f" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3"
I20260807 20:08:12.026175 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a63b14addb7749428ea0e17c792e4d0f" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
I20260807 20:08:12.026225 5260 raft_consensus.cc:3037] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:12.026245 5113 raft_consensus.cc:3037] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:12.027040 5113 raft_consensus.cc:2454] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 1.
I20260807 20:08:12.027082 5260 raft_consensus.cc:2454] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 1.
I20260807 20:08:12.027232 5338 leader_election.cc:302] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:12.027333 5484 raft_consensus.cc:2781] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:12.027442 5484 raft_consensus.cc:686] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 LEADER]: Becoming Leader. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Running, Role: LEADER
I20260807 20:08:12.027542 5484 consensus_queue.cc:237] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.027966 5502 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: TxnStatusTablet state changed. Reason: New leader 2883d7ebb37546619fd83e85f9c2ca14. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.027968 5485 consensus_queue.cc:1035] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:12.028155 5502 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:12.028419 4948 catalog_manager.cc:5495] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:12.028870 5504 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 20:08:12.029004 5504 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 20:08:12.029752 5485 consensus_queue.cc:1035] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:12.086647 5495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.083855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.0.0.1:59326)
0807 20:08:12.083946 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:12.083973 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:12.084025 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:12.084234 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:12.084236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:12.084237 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:12.084294 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.084296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.085045 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.085046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.085642 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.085644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.086029 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.086030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.086102 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.086120 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.086442 (+ 322us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.086453 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 20:08:12.086510 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 20:08:12.086563 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 20:08:12.086627 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.083932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59326 (local address 127.3.15.131:42239)
0807 20:08:12.084000 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.084002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.084013 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:12.084120 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.084121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.084140 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.084225 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:12.084373 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.084957 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.085734 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.085951 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.086030 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.086256 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.086370 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.086463 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.086515 (+ 52us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.086540 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:08:12.088457 5113 raft_consensus.cc:1261] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Refusing update from remote peer 2883d7ebb37546619fd83e85f9c2ca14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:12.088528 5260 raft_consensus.cc:1261] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Refusing update from remote peer 2883d7ebb37546619fd83e85f9c2ca14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:12.088768 5502 consensus_queue.cc:1035] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:12.088963 5484 consensus_queue.cc:1035] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:12.090094 5486 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: TxnStatusTablet state changed. Reason: New leader 2883d7ebb37546619fd83e85f9c2ca14. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.090204 5486 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:12.090795 5507 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: TxnStatusTablet state changed. Reason: New leader 2883d7ebb37546619fd83e85f9c2ca14. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.090910 5507 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:12.091436 5484 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.091542 5484 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:12.091673 5484 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.091744 5484 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2883d7ebb37546619fd83e85f9c2ca14: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:12.092200 5506 mvcc.cc:205] Tried to move back new op lower bound from 7316001964391940096 to 7316001964145483776. Current Snapshot: MvccSnapshot[applied={T|T < 7316001964391940096}]
I20260807 20:08:12.092547 5486 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.092674 5486 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 2c3f0ae4a45d414cbcdfd396dc03c16b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:12.093640 5507 tablet_replica.cc:439] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } } }
I20260807 20:08:12.093855 5507 tablet_replica.cc:442] T a63b14addb7749428ea0e17c792e4d0f P 90edb56383614ba18f33dca3c81243f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:12.097504 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.094379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.3.15.130:34127)
0807 20:08:12.094446 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.094448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.094545 (+ 97us) server_negotiation.cc:407] Connection header received
0807 20:08:12.094597 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.094598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.094617 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.094696 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:12.094806 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.095562 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.096473 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.096802 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.096877 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.097166 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.097266 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.097330 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.097429 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.097452 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:12.102851 5522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.100111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.3.15.130:34823)
0807 20:08:12.100292 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:12.100305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:12.100368 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:12.100503 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:12.100504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:12.100505 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:12.100556 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.100557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.101430 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.101430 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.101981 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.101982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.102448 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.102450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.102501 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.102515 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.102715 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.102717 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:08:12.102747 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 20:08:12.102783 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":117,"threads_started":1}
I20260807 20:08:12.102864 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.100244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:34823 (local address 127.3.15.131:42239)
0807 20:08:12.100294 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.100296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.100378 (+ 82us) server_negotiation.cc:407] Connection header received
0807 20:08:12.100408 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.100409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.100423 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.100496 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:12.100618 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.101348 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.102083 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.102368 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.102449 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.102611 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.102648 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.102709 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.102763 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.102788 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:08:12.114089 5474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.111051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:39279 (local address 127.3.15.130:34127)
0807 20:08:12.111101 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.111103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.111271 (+ 168us) server_negotiation.cc:407] Connection header received
0807 20:08:12.111301 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.111302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.111316 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.111378 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:12.111481 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.112230 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.113435 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.113635 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.113685 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.113892 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.113919 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.113957 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.114017 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.114034 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:08:12.114171 5525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.110986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:34127 (local address 127.3.15.131:39279)
0807 20:08:12.111157 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:12.111169 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:12.111231 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:12.111379 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:12.111380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:12.111381 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:12.111438 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.111440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.112336 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.112337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.113336 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.113340 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.113716 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.113717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.113789 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.113806 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.113962 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.113963 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:08:12.114015 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:08:12.114069 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260807 20:08:12.122991 4947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:12.123131 4947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:12.123982 4947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Sending DeleteTablet for 3 replicas of tablet a1f4ef19f40648a28af0e77145a808ac
I20260807 20:08:12.124588 5238 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f4ef19f40648a28af0e77145a808ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:12 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:49126
I20260807 20:08:12.124588 5386 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f4ef19f40648a28af0e77145a808ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:12 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59312
I20260807 20:08:12.124851 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet a1f4ef19f40648a28af0e77145a808ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:12 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:46804
I20260807 20:08:12.125103 5532 tablet_replica.cc:330] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: stopping tablet replica
I20260807 20:08:12.125092 4947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36042:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:12.125258 4947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:12.125445 5532 raft_consensus.cc:2229] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:12.125578 5532 raft_consensus.cc:2258] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:12.126055 5532 ts_tablet_manager.cc:1905] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:12.126801 5386 tablet_service.cc:1467] Processing CreateTablet for tablet 90cc05355df7486eb8aedd80b09a41fe (DEFAULT_TABLE table=test-table [id=e579fbfcd84f408da48c97d1b6390f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:12.126940 5386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90cc05355df7486eb8aedd80b09a41fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:12.128315 5478 tablet_bootstrap.cc:492] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Bootstrap starting.
I20260807 20:08:12.128353 5532 ts_tablet_manager.cc:1918] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:12.128553 5532 log.cc:1196] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/a1f4ef19f40648a28af0e77145a808ac
I20260807 20:08:12.128966 5532 ts_tablet_manager.cc:1939] T a1f4ef19f40648a28af0e77145a808ac P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting consensus metadata
I20260807 20:08:12.129179 5478 tablet_bootstrap.cc:654] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:12.129806 4934 catalog_manager.cc:4841] TS 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013): tablet a1f4ef19f40648a28af0e77145a808ac (table test-table [id=a5a6f36273374a5ab59482bc2b1b9267]) successfully deleted
I20260807 20:08:12.130033 5092 tablet_service.cc:1467] Processing CreateTablet for tablet 90cc05355df7486eb8aedd80b09a41fe (DEFAULT_TABLE table=test-table [id=e579fbfcd84f408da48c97d1b6390f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:12.130158 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90cc05355df7486eb8aedd80b09a41fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:12.131451 5479 tablet_bootstrap.cc:492] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Bootstrap starting.
I20260807 20:08:12.131553 5238 tablet_service.cc:1467] Processing CreateTablet for tablet 90cc05355df7486eb8aedd80b09a41fe (DEFAULT_TABLE table=test-table [id=e579fbfcd84f408da48c97d1b6390f3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:12.131673 5238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90cc05355df7486eb8aedd80b09a41fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:12.131877 5479 tablet_bootstrap.cc:654] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:12.133071 5477 tablet_bootstrap.cc:492] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Bootstrap starting.
I20260807 20:08:12.133141 5478 tablet_bootstrap.cc:492] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: No bootstrap required, opened a new log
I20260807 20:08:12.133183 5478 ts_tablet_manager.cc:1397] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260807 20:08:12.133337 5478 raft_consensus.cc:348] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.133412 5478 raft_consensus.cc:374] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:12.133440 5478 raft_consensus.cc:729] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Initialized, Role: FOLLOWER
I20260807 20:08:12.133514 5477 tablet_bootstrap.cc:654] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:12.133498 5478 consensus_queue.cc:260] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.133607 5478 ts_tablet_manager.cc:1428] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:12.133991 5531 tablet_replica.cc:330] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: stopping tablet replica
I20260807 20:08:12.134174 5531 raft_consensus.cc:2229] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:12.134327 5531 raft_consensus.cc:2258] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:12.134536 5530 tablet_replica.cc:330] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: stopping tablet replica
I20260807 20:08:12.134680 5530 raft_consensus.cc:2229] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:12.134774 5530 raft_consensus.cc:2258] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:12.135149 5531 ts_tablet_manager.cc:1905] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:12.135186 5530 ts_tablet_manager.cc:1905] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:12.137131 5530 ts_tablet_manager.cc:1918] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:12.137262 5530 log.cc:1196] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/a1f4ef19f40648a28af0e77145a808ac
I20260807 20:08:12.137302 5531 ts_tablet_manager.cc:1918] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:12.137384 5531 log.cc:1196] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/a1f4ef19f40648a28af0e77145a808ac
I20260807 20:08:12.137669 5531 ts_tablet_manager.cc:1939] T a1f4ef19f40648a28af0e77145a808ac P 90edb56383614ba18f33dca3c81243f3: Deleting consensus metadata
I20260807 20:08:12.137781 5530 ts_tablet_manager.cc:1939] T a1f4ef19f40648a28af0e77145a808ac P 2883d7ebb37546619fd83e85f9c2ca14: Deleting consensus metadata
I20260807 20:08:12.138084 4931 catalog_manager.cc:4841] TS 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127): tablet a1f4ef19f40648a28af0e77145a808ac (table test-table [id=a5a6f36273374a5ab59482bc2b1b9267]) successfully deleted
I20260807 20:08:12.138269 4934 catalog_manager.cc:4841] TS 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239): tablet a1f4ef19f40648a28af0e77145a808ac (table test-table [id=a5a6f36273374a5ab59482bc2b1b9267]) successfully deleted
I20260807 20:08:12.141230 5477 tablet_bootstrap.cc:492] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: No bootstrap required, opened a new log
I20260807 20:08:12.141287 5477 ts_tablet_manager.cc:1397] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260807 20:08:12.141377 5479 tablet_bootstrap.cc:492] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: No bootstrap required, opened a new log
I20260807 20:08:12.141448 5479 ts_tablet_manager.cc:1397] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260807 20:08:12.141435 5477 raft_consensus.cc:348] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.141503 5477 raft_consensus.cc:374] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:12.141525 5477 raft_consensus.cc:729] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90edb56383614ba18f33dca3c81243f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:12.141567 5477 consensus_queue.cc:260] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.141599 5479 raft_consensus.cc:348] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.141673 5479 raft_consensus.cc:374] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:12.141670 5477 ts_tablet_manager.cc:1428] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:12.141695 5479 raft_consensus.cc:729] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3f0ae4a45d414cbcdfd396dc03c16b, State: Initialized, Role: FOLLOWER
I20260807 20:08:12.141736 5479 consensus_queue.cc:260] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.141842 5479 ts_tablet_manager.cc:1428] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:12.251657 5502 raft_consensus.cc:482] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:12.251770 5502 raft_consensus.cc:504] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.251984 5502 leader_election.cc:288] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:12.252221 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc05355df7486eb8aedd80b09a41fe" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" is_pre_election: true
I20260807 20:08:12.252255 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc05355df7486eb8aedd80b09a41fe" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3" is_pre_election: true
I20260807 20:08:12.252336 5113 raft_consensus.cc:2454] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 0.
I20260807 20:08:12.252362 5260 raft_consensus.cc:2454] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 0.
I20260807 20:08:12.252534 5338 leader_election.cc:302] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:12.252664 5502 raft_consensus.cc:2781] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:12.252717 5502 raft_consensus.cc:482] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:12.252767 5502 raft_consensus.cc:3037] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:12.253607 5502 raft_consensus.cc:504] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.253769 5502 leader_election.cc:288] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:12.253943 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc05355df7486eb8aedd80b09a41fe" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3"
I20260807 20:08:12.253952 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90cc05355df7486eb8aedd80b09a41fe" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
I20260807 20:08:12.254038 5113 raft_consensus.cc:3037] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:12.254040 5260 raft_consensus.cc:3037] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:12.254730 5113 raft_consensus.cc:2454] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 1.
I20260807 20:08:12.254801 5260 raft_consensus.cc:2454] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 1.
I20260807 20:08:12.254948 5338 leader_election.cc:302] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:12.255100 5502 raft_consensus.cc:2781] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:12.255157 5502 raft_consensus.cc:686] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 LEADER]: Becoming Leader. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Running, Role: LEADER
I20260807 20:08:12.255213 5502 consensus_queue.cc:237] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:12.255895 4948 catalog_manager.cc:5495] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:12.284852 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.282594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59340 (local address 127.3.15.131:42239)
0807 20:08:12.282645 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.282647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.282657 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:12.282716 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.282717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.282728 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.282795 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:12.282906 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.283387 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.284151 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.284348 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.284424 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.284557 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.284596 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.284655 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.284719 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.284738 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:08:12.289203 5476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.287124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:47189 (local address 127.3.15.131:42239)
0807 20:08:12.287158 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:12.287160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:12.287169 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:12.287234 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:12.287235 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:12.287242 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:12.287278 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:12.287351 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.287783 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.288525 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.288718 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.288807 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.288948 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.288978 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.289045 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:12.289103 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 20:08:12.289120 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 20:08:12.289207 5525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:12.287068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.3.15.131:47189)
0807 20:08:12.287127 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:12.287143 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:12.287176 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:12.287280 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:12.287280 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:12.287281 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:12.287317 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.287318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.287862 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.287863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.288429 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:12.288431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:12.288809 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:12.288809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:12.288858 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:12.288864 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:12.289044 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:12.289046 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:08:12.289099 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:08:12.289145 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:08:12.291363 5260 raft_consensus.cc:1261] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Refusing update from remote peer 2883d7ebb37546619fd83e85f9c2ca14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:12.291388 5113 raft_consensus.cc:1261] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Refusing update from remote peer 2883d7ebb37546619fd83e85f9c2ca14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:12.291661 5523 consensus_queue.cc:1035] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:12.291769 5484 consensus_queue.cc:1035] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:12.293196 5511 mvcc.cc:205] Tried to move back new op lower bound from 7316001965224062976 to 7316001965077807104. Current Snapshot: MvccSnapshot[applied={T|T < 7316001965224062976}]
I20260807 20:08:12.293720 5509 mvcc.cc:205] Tried to move back new op lower bound from 7316001965224062976 to 7316001965077807104. Current Snapshot: MvccSnapshot[applied={T|T < 7316001965224062976}]
W20260807 20:08:12.297789 5509 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 20:08:13.816515 4947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:13.816632 4947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36042:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:13.817528 4947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Sending DeleteTablet for 3 replicas of tablet 90cc05355df7486eb8aedd80b09a41fe
I20260807 20:08:13.817791 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet 90cc05355df7486eb8aedd80b09a41fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:13 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:46804
I20260807 20:08:13.817833 5238 tablet_service.cc:1514] Processing DeleteTablet for tablet 90cc05355df7486eb8aedd80b09a41fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:13 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:49126
I20260807 20:08:13.817989 5386 tablet_service.cc:1514] Processing DeleteTablet for tablet 90cc05355df7486eb8aedd80b09a41fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:13 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59312
I20260807 20:08:13.818091 5564 tablet_replica.cc:330] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: stopping tablet replica
I20260807 20:08:13.818149 5564 raft_consensus.cc:2229] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:13.818187 5564 raft_consensus.cc:2258] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:13.818522 5564 ts_tablet_manager.cc:1905] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:13.820384 5564 ts_tablet_manager.cc:1918] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:08:13.820461 5566 tablet_replica.cc:330] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: stopping tablet replica
I20260807 20:08:13.820529 5564 log.cc:1196] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/90cc05355df7486eb8aedd80b09a41fe
I20260807 20:08:13.820554 5566 raft_consensus.cc:2229] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:13.820674 5566 raft_consensus.cc:2258] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:13.820917 5564 ts_tablet_manager.cc:1939] T 90cc05355df7486eb8aedd80b09a41fe P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting consensus metadata
I20260807 20:08:13.820990 5566 ts_tablet_manager.cc:1905] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:13.821314 4934 catalog_manager.cc:4841] TS 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013): tablet 90cc05355df7486eb8aedd80b09a41fe (table test-table [id=e579fbfcd84f408da48c97d1b6390f3a]) successfully deleted
I20260807 20:08:13.822926 5566 ts_tablet_manager.cc:1918] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:08:13.823004 5566 log.cc:1196] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/90cc05355df7486eb8aedd80b09a41fe
I20260807 20:08:13.823148 5565 tablet_replica.cc:330] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: stopping tablet replica
I20260807 20:08:13.823271 5566 ts_tablet_manager.cc:1939] T 90cc05355df7486eb8aedd80b09a41fe P 2883d7ebb37546619fd83e85f9c2ca14: Deleting consensus metadata
I20260807 20:08:13.823276 5565 raft_consensus.cc:2229] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:13.823341 5565 raft_consensus.cc:2258] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:13.823577 5565 ts_tablet_manager.cc:1905] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:13.823668 4934 catalog_manager.cc:4841] TS 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239): tablet 90cc05355df7486eb8aedd80b09a41fe (table test-table [id=e579fbfcd84f408da48c97d1b6390f3a]) successfully deleted
I20260807 20:08:13.825490 5565 ts_tablet_manager.cc:1918] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:08:13.825564 5565 log.cc:1196] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/90cc05355df7486eb8aedd80b09a41fe
I20260807 20:08:13.825825 5565 ts_tablet_manager.cc:1939] T 90cc05355df7486eb8aedd80b09a41fe P 90edb56383614ba18f33dca3c81243f3: Deleting consensus metadata
I20260807 20:08:13.826211 4931 catalog_manager.cc:4841] TS 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127): tablet 90cc05355df7486eb8aedd80b09a41fe (table test-table [id=e579fbfcd84f408da48c97d1b6390f3a]) successfully deleted
Aug 07 20:08:13 dist-test-slave-1x4g krb5kdc[4897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133293, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:08:13 dist-test-slave-1x4g krb5kdc[4897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133293, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:13.843909 5575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:13.836739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.3.15.190:38459)
0807 20:08:13.836906 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:13.836908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:13.837034 (+ 126us) server_negotiation.cc:407] Connection header received
0807 20:08:13.837084 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:13.837085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:13.837095 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:13.837258 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:13.837831 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:13.838375 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:13.839087 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:13.839280 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:13.839334 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:13.841611 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:13.841613 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:13.841613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:13.841620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:13.842949 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:13.843264 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:13.843265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:13.843265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:13.843318 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:13.843482 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:13.843482 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:13.843483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:13.843584 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:13.843669 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:13.843755 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 20:08:13.843823 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260807 20:08:13.845629 4947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:13.845818 4947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:13.847394 5386 tablet_service.cc:1467] Processing CreateTablet for tablet ca0dbbfcc2a54311a6d284ff86c54f6c (DEFAULT_TABLE table=test-table [id=15441a43a4b04999a39d0608fd7dc54d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:13.847359 5238 tablet_service.cc:1467] Processing CreateTablet for tablet ca0dbbfcc2a54311a6d284ff86c54f6c (DEFAULT_TABLE table=test-table [id=15441a43a4b04999a39d0608fd7dc54d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:13.847550 5386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0dbbfcc2a54311a6d284ff86c54f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:13.847564 5238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0dbbfcc2a54311a6d284ff86c54f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:13.848843 5576 tablet_bootstrap.cc:492] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Bootstrap starting.
I20260807 20:08:13.849004 5092 tablet_service.cc:1467] Processing CreateTablet for tablet ca0dbbfcc2a54311a6d284ff86c54f6c (DEFAULT_TABLE table=test-table [id=15441a43a4b04999a39d0608fd7dc54d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:13.849139 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca0dbbfcc2a54311a6d284ff86c54f6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:13.849336 5576 tablet_bootstrap.cc:654] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:13.850004 5576 tablet_bootstrap.cc:492] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: No bootstrap required, opened a new log
I20260807 20:08:13.850057 5576 ts_tablet_manager.cc:1397] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:13.850170 5576 raft_consensus.cc:348] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:13.850237 5576 raft_consensus.cc:374] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:13.850257 5576 raft_consensus.cc:729] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Initialized, Role: FOLLOWER
I20260807 20:08:13.850313 5576 consensus_queue.cc:260] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:13.850414 5576 ts_tablet_manager.cc:1428] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:13.850687 5579 tablet_bootstrap.cc:492] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Bootstrap starting.
I20260807 20:08:13.850813 5577 tablet_bootstrap.cc:492] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Bootstrap starting.
I20260807 20:08:13.851138 5579 tablet_bootstrap.cc:654] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:13.851223 5577 tablet_bootstrap.cc:654] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:13.851778 5579 tablet_bootstrap.cc:492] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: No bootstrap required, opened a new log
I20260807 20:08:13.851846 5579 ts_tablet_manager.cc:1397] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:13.851890 5577 tablet_bootstrap.cc:492] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: No bootstrap required, opened a new log
I20260807 20:08:13.851950 5577 ts_tablet_manager.cc:1397] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:13.852041 5579 raft_consensus.cc:348] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:13.852065 5577 raft_consensus.cc:348] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:13.852128 5577 raft_consensus.cc:374] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:13.852130 5579 raft_consensus.cc:374] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:13.852156 5579 raft_consensus.cc:729] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3f0ae4a45d414cbcdfd396dc03c16b, State: Initialized, Role: FOLLOWER
I20260807 20:08:13.852142 5577 raft_consensus.cc:729] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90edb56383614ba18f33dca3c81243f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:13.852209 5579 consensus_queue.cc:260] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:13.852236 5577 consensus_queue.cc:260] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:13.852397 5579 ts_tablet_manager.cc:1428] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:13.852553 5577 ts_tablet_manager.cc:1428] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:14.186218 5550 raft_consensus.cc:482] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:14.186347 5550 raft_consensus.cc:504] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.186565 5550 leader_election.cc:288] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:14.186826 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0dbbfcc2a54311a6d284ff86c54f6c" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3" is_pre_election: true
I20260807 20:08:14.186826 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0dbbfcc2a54311a6d284ff86c54f6c" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" is_pre_election: true
I20260807 20:08:14.186964 5260 raft_consensus.cc:2454] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 0.
I20260807 20:08:14.186980 5113 raft_consensus.cc:2454] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 0.
I20260807 20:08:14.187166 5338 leader_election.cc:302] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:14.187278 5550 raft_consensus.cc:2781] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:14.187338 5550 raft_consensus.cc:482] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:14.187364 5550 raft_consensus.cc:3037] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:14.188527 5550 raft_consensus.cc:504] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.188714 5550 leader_election.cc:288] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 election: Requested vote from peers 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013), 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127)
I20260807 20:08:14.188892 5113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0dbbfcc2a54311a6d284ff86c54f6c" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b"
I20260807 20:08:14.188931 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca0dbbfcc2a54311a6d284ff86c54f6c" candidate_uuid: "2883d7ebb37546619fd83e85f9c2ca14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3"
I20260807 20:08:14.188982 5113 raft_consensus.cc:3037] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:14.189002 5260 raft_consensus.cc:3037] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:14.189743 5113 raft_consensus.cc:2454] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 1.
I20260807 20:08:14.189743 5260 raft_consensus.cc:2454] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2883d7ebb37546619fd83e85f9c2ca14 in term 1.
I20260807 20:08:14.189944 5338 leader_election.cc:302] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:14.190042 5550 raft_consensus.cc:2781] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:14.190096 5550 raft_consensus.cc:686] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 LEADER]: Becoming Leader. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Running, Role: LEADER
I20260807 20:08:14.190142 5550 consensus_queue.cc:237] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.190815 4948 catalog_manager.cc:5495] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "2883d7ebb37546619fd83e85f9c2ca14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:14.207027 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.204371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59344 (local address 127.3.15.131:42239)
0807 20:08:14.204544 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:14.204545 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:14.204551 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:08:14.204570 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:14.204571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:14.204580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:14.204864 (+ 284us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:14.205018 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.205516 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.206301 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.206510 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.206585 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.206745 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:14.206786 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:14.206848 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:14.206911 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 20:08:14.206936 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
I20260807 20:08:14.209554 5113 raft_consensus.cc:1261] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Refusing update from remote peer 2883d7ebb37546619fd83e85f9c2ca14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:14.209592 5260 raft_consensus.cc:1261] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Refusing update from remote peer 2883d7ebb37546619fd83e85f9c2ca14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:14.209970 5545 consensus_queue.cc:1035] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:14.210109 5550 consensus_queue.cc:1035] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:14.220273 4948 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:14.220395 4948 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:14.221076 4948 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Sending DeleteTablet for 3 replicas of tablet ca0dbbfcc2a54311a6d284ff86c54f6c
I20260807 20:08:14.221359 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0dbbfcc2a54311a6d284ff86c54f6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:14 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:46804
I20260807 20:08:14.221357 5238 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0dbbfcc2a54311a6d284ff86c54f6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:14 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:49126
I20260807 20:08:14.221467 5564 tablet_replica.cc:330] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: stopping tablet replica
I20260807 20:08:14.221481 5385 tablet_service.cc:1514] Processing DeleteTablet for tablet ca0dbbfcc2a54311a6d284ff86c54f6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:14 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59312
I20260807 20:08:14.221494 5565 tablet_replica.cc:330] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: stopping tablet replica
I20260807 20:08:14.221513 5564 raft_consensus.cc:2229] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:14.221555 5564 raft_consensus.cc:2258] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:14.221568 5565 raft_consensus.cc:2229] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:14.221613 5565 raft_consensus.cc:2258] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:14.221643 5566 tablet_replica.cc:330] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: stopping tablet replica
I20260807 20:08:14.221696 5566 raft_consensus.cc:2229] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:14.221755 5566 raft_consensus.cc:2258] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:14.221930 5564 ts_tablet_manager.cc:1905] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:14.221988 5565 ts_tablet_manager.cc:1905] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:14.222150 5566 ts_tablet_manager.cc:1905] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:14.223443 4948 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:14.223621 4948 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:14.223762 5565 ts_tablet_manager.cc:1918] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:14.223778 5564 ts_tablet_manager.cc:1918] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:14.223830 5565 log.cc:1196] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/ca0dbbfcc2a54311a6d284ff86c54f6c
I20260807 20:08:14.223830 5564 log.cc:1196] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/ca0dbbfcc2a54311a6d284ff86c54f6c
I20260807 20:08:14.223857 5566 ts_tablet_manager.cc:1918] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:14.223908 5566 log.cc:1196] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/ca0dbbfcc2a54311a6d284ff86c54f6c
I20260807 20:08:14.224089 5564 ts_tablet_manager.cc:1939] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2c3f0ae4a45d414cbcdfd396dc03c16b: Deleting consensus metadata
I20260807 20:08:14.224102 5566 ts_tablet_manager.cc:1939] T ca0dbbfcc2a54311a6d284ff86c54f6c P 2883d7ebb37546619fd83e85f9c2ca14: Deleting consensus metadata
I20260807 20:08:14.224110 5565 ts_tablet_manager.cc:1939] T ca0dbbfcc2a54311a6d284ff86c54f6c P 90edb56383614ba18f33dca3c81243f3: Deleting consensus metadata
I20260807 20:08:14.224442 4934 catalog_manager.cc:4841] TS 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013): tablet ca0dbbfcc2a54311a6d284ff86c54f6c (table test-table [id=15441a43a4b04999a39d0608fd7dc54d]) successfully deleted
I20260807 20:08:14.224510 4931 catalog_manager.cc:4841] TS 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127): tablet ca0dbbfcc2a54311a6d284ff86c54f6c (table test-table [id=15441a43a4b04999a39d0608fd7dc54d]) successfully deleted
I20260807 20:08:14.224519 4934 catalog_manager.cc:4841] TS 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239): tablet ca0dbbfcc2a54311a6d284ff86c54f6c (table test-table [id=15441a43a4b04999a39d0608fd7dc54d]) successfully deleted
I20260807 20:08:14.225557 5238 tablet_service.cc:1467] Processing CreateTablet for tablet 648655294c1c42339edcf74d72f4f4de (DEFAULT_TABLE table=test-table [id=a075388a10ca4b1996157a538f9060bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:14.225557 5092 tablet_service.cc:1467] Processing CreateTablet for tablet 648655294c1c42339edcf74d72f4f4de (DEFAULT_TABLE table=test-table [id=a075388a10ca4b1996157a538f9060bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:14.225595 5385 tablet_service.cc:1467] Processing CreateTablet for tablet 648655294c1c42339edcf74d72f4f4de (DEFAULT_TABLE table=test-table [id=a075388a10ca4b1996157a538f9060bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:14.225678 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648655294c1c42339edcf74d72f4f4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:14.225678 5238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648655294c1c42339edcf74d72f4f4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:14.225723 5385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 648655294c1c42339edcf74d72f4f4de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:14.226892 5577 tablet_bootstrap.cc:492] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Bootstrap starting.
I20260807 20:08:14.226886 5579 tablet_bootstrap.cc:492] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b: Bootstrap starting.
I20260807 20:08:14.227072 5576 tablet_bootstrap.cc:492] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Bootstrap starting.
I20260807 20:08:14.227247 5577 tablet_bootstrap.cc:654] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:14.227407 5579 tablet_bootstrap.cc:654] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:14.227415 5576 tablet_bootstrap.cc:654] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:14.227840 5577 tablet_bootstrap.cc:492] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: No bootstrap required, opened a new log
I20260807 20:08:14.227896 5577 ts_tablet_manager.cc:1397] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:14.228029 5577 raft_consensus.cc:348] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.228106 5579 tablet_bootstrap.cc:492] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b: No bootstrap required, opened a new log
I20260807 20:08:14.228116 5577 raft_consensus.cc:374] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:14.228144 5577 raft_consensus.cc:729] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90edb56383614ba18f33dca3c81243f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:14.228157 5579 ts_tablet_manager.cc:1397] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:14.228190 5577 consensus_queue.cc:260] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.228283 5577 ts_tablet_manager.cc:1428] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:14.228272 5579 raft_consensus.cc:348] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.228323 5579 raft_consensus.cc:374] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:14.228395 5579 raft_consensus.cc:729] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3f0ae4a45d414cbcdfd396dc03c16b, State: Initialized, Role: FOLLOWER
I20260807 20:08:14.228451 5579 consensus_queue.cc:260] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.228559 5576 tablet_bootstrap.cc:492] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: No bootstrap required, opened a new log
I20260807 20:08:14.228572 5579 ts_tablet_manager.cc:1428] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:14.228609 5576 ts_tablet_manager.cc:1397] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:14.228704 5576 raft_consensus.cc:348] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.228778 5576 raft_consensus.cc:374] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:14.228822 5576 raft_consensus.cc:729] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2883d7ebb37546619fd83e85f9c2ca14, State: Initialized, Role: FOLLOWER
I20260807 20:08:14.228868 5576 consensus_queue.cc:260] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.228965 5576 ts_tablet_manager.cc:1428] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:14.449515 5582 raft_consensus.cc:482] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:14.449676 5582 raft_consensus.cc:504] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.450101 5582 leader_election.cc:288] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127), 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239)
I20260807 20:08:14.453392 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.450494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:48693 (local address 127.3.15.131:42239)
0807 20:08:14.450542 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:14.450545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:14.450729 (+ 184us) server_negotiation.cc:407] Connection header received
0807 20:08:14.450808 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:14.450809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:14.450826 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:14.450892 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:14.451064 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.451551 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.452543 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.453026 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.453111 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.453122 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:14.453277 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 20:08:14.453315 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:08:14.453423 5596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.450304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.3.15.129:48693)
0807 20:08:14.450468 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:14.450481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:14.450732 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:14.450896 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:14.450897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:14.450899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:14.450983 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.450985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.451644 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.451645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.452449 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.452451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.453114 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.453116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.453192 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.453196 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:14.453270 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:08:14.453315 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":80,"threads_started":1}
I20260807 20:08:14.453486 5597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.450392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:57045 (local address 127.3.15.130:34127)
0807 20:08:14.450732 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:14.450734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:14.450774 (+ 40us) server_negotiation.cc:407] Connection header received
0807 20:08:14.450877 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:14.450878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:14.450890 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:14.451097 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:14.451271 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.451748 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.452657 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.453100 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.453187 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.453195 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:14.453362 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 20:08:14.453405 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":158,"threads_started":1}
I20260807 20:08:14.453692 5595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.450322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:34127 (local address 127.3.15.129:57045)
0807 20:08:14.450509 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:14.450520 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:14.450797 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:14.451100 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:14.451102 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:14.451104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:14.451178 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.451182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.451855 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.451856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.452564 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.452566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.453198 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.453200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.453286 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.453293 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:08:14.453357 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:08:14.453400 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
I20260807 20:08:14.453985 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648655294c1c42339edcf74d72f4f4de" candidate_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3" is_pre_election: true
I20260807 20:08:14.453985 5406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648655294c1c42339edcf74d72f4f4de" candidate_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2883d7ebb37546619fd83e85f9c2ca14" is_pre_election: true
I20260807 20:08:14.454093 5260 raft_consensus.cc:2454] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3f0ae4a45d414cbcdfd396dc03c16b in term 0.
I20260807 20:08:14.454100 5406 raft_consensus.cc:2454] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3f0ae4a45d414cbcdfd396dc03c16b in term 0.
I20260807 20:08:14.454329 5041 leader_election.cc:302] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:14.454474 5582 raft_consensus.cc:2781] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:14.454512 5582 raft_consensus.cc:482] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:14.454541 5582 raft_consensus.cc:3037] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:14.455281 5582 raft_consensus.cc:504] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.455415 5582 leader_election.cc:288] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [CANDIDATE]: Term 1 election: Requested vote from peers 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127), 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239)
I20260807 20:08:14.455588 5406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648655294c1c42339edcf74d72f4f4de" candidate_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2883d7ebb37546619fd83e85f9c2ca14"
I20260807 20:08:14.455595 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "648655294c1c42339edcf74d72f4f4de" candidate_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90edb56383614ba18f33dca3c81243f3"
I20260807 20:08:14.455682 5260 raft_consensus.cc:3037] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:14.455687 5406 raft_consensus.cc:3037] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:14.456565 5260 raft_consensus.cc:2454] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3f0ae4a45d414cbcdfd396dc03c16b in term 1.
I20260807 20:08:14.456573 5406 raft_consensus.cc:2454] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3f0ae4a45d414cbcdfd396dc03c16b in term 1.
I20260807 20:08:14.456763 5041 leader_election.cc:302] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2883d7ebb37546619fd83e85f9c2ca14, 2c3f0ae4a45d414cbcdfd396dc03c16b; no voters:
I20260807 20:08:14.456907 5582 raft_consensus.cc:2781] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:14.457090 5582 raft_consensus.cc:686] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [term 1 LEADER]: Becoming Leader. State: Replica: 2c3f0ae4a45d414cbcdfd396dc03c16b, State: Running, Role: LEADER
I20260807 20:08:14.457177 5582 consensus_queue.cc:237] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } }
I20260807 20:08:14.457990 4947 catalog_manager.cc:5495] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129). New cstate: current_term: 1 leader_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3f0ae4a45d414cbcdfd396dc03c16b" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 } health_report { overall_health: UNKNOWN } } }
I20260807 20:08:14.517784 5600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.514780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46806 (local address 127.3.15.129:46013)
0807 20:08:14.514991 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:14.514992 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:14.514999 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:14.515036 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:14.515037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:14.515058 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:14.515307 (+ 249us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:14.515473 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.516098 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.516906 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.517149 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.517239 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.517396 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:14.517521 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:14.517613 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:14.517642 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 20:08:14.517662 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":122,"threads_started":1}
I20260807 20:08:14.522897 5586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.520278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:57189 (local address 127.3.15.131:42239)
0807 20:08:14.520343 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:14.520349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:14.520660 (+ 311us) server_negotiation.cc:407] Connection header received
0807 20:08:14.520710 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:14.520711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:14.520720 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:14.520760 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:14.520948 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.521411 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.522109 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.522290 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.522341 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.522636 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:14.522665 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:14.522704 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:14.522815 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 20:08:14.522833 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:08:14.523044 5603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.520215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42239 (local address 127.3.15.129:57189)
0807 20:08:14.520402 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:14.520430 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:14.520655 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:14.520816 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:14.520817 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:14.520819 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:14.520888 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.520889 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.521476 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.521477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.522030 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.522032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.522401 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.522403 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.522539 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.522555 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:14.522749 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:14.522752 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:14.522804 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:08:14.522844 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
I20260807 20:08:14.525565 5260 raft_consensus.cc:1261] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Refusing update from remote peer 2c3f0ae4a45d414cbcdfd396dc03c16b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:14.525616 5406 raft_consensus.cc:1261] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Refusing update from remote peer 2c3f0ae4a45d414cbcdfd396dc03c16b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:14.525832 5598 consensus_queue.cc:1035] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "90edb56383614ba18f33dca3c81243f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:14.525978 5582 consensus_queue.cc:1035] T 648655294c1c42339edcf74d72f4f4de P 2c3f0ae4a45d414cbcdfd396dc03c16b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2883d7ebb37546619fd83e85f9c2ca14" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:14.533780 5600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.531154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:58513 (local address 127.3.15.129:46013)
0807 20:08:14.531211 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:14.531213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:14.531623 (+ 410us) server_negotiation.cc:407] Connection header received
0807 20:08:14.531647 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:14.531648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:14.531657 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:14.531694 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:14.531783 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.532232 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.533271 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.533470 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.533526 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.533630 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:14.533654 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:14.533687 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:14.533717 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 20:08:14.533732 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:08:14.533804 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:14.531125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46013 (local address 127.3.15.131:58513)
0807 20:08:14.531424 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:14.531433 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:14.531589 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:14.531697 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:14.531697 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:14.531698 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:14.531745 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.531747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.532297 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.532297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.533194 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:14.533198 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:14.533533 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:14.533534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:14.533583 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:14.533587 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:14.533686 (+ 99us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:14.533686 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 20:08:14.533716 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 20:08:14.533750 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":253,"threads_started":1}
W20260807 20:08:14.534432 5546 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 20:08:16.066368 4946 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:16.066483 4946 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:16.067523 4946 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7649da9749194a5494794274f6176533: Sending DeleteTablet for 3 replicas of tablet 648655294c1c42339edcf74d72f4f4de
I20260807 20:08:16.067848 5385 tablet_service.cc:1514] Processing DeleteTablet for tablet 648655294c1c42339edcf74d72f4f4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:16 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59312
I20260807 20:08:16.067880 5238 tablet_service.cc:1514] Processing DeleteTablet for tablet 648655294c1c42339edcf74d72f4f4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:16 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:49126
I20260807 20:08:16.067884 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet 648655294c1c42339edcf74d72f4f4de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:16 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:46804
I20260807 20:08:16.068058 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 5022
I20260807 20:08:16.070102 5637 tablet_replica.cc:330] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: stopping tablet replica
I20260807 20:08:16.070188 5637 raft_consensus.cc:2229] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:16.070240 5637 raft_consensus.cc:2258] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:16.070505 5637 ts_tablet_manager.cc:1905] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:16.071770 5635 tablet_replica.cc:330] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: stopping tablet replica
I20260807 20:08:16.071848 5635 raft_consensus.cc:2229] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:16.071887 5635 raft_consensus.cc:2258] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:16.072221 5635 ts_tablet_manager.cc:1905] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:16.072569 5637 ts_tablet_manager.cc:1918] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:08:16.072656 5637 log.cc:1196] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/648655294c1c42339edcf74d72f4f4de
I20260807 20:08:16.073019 5637 ts_tablet_manager.cc:1939] T 648655294c1c42339edcf74d72f4f4de P 2883d7ebb37546619fd83e85f9c2ca14: Deleting consensus metadata
I20260807 20:08:16.073417 4934 catalog_manager.cc:4841] TS 2883d7ebb37546619fd83e85f9c2ca14 (127.3.15.131:42239): tablet 648655294c1c42339edcf74d72f4f4de (table test-table [id=a075388a10ca4b1996157a538f9060bb]) successfully deleted
I20260807 20:08:16.073695 5635 ts_tablet_manager.cc:1918] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 20:08:16.073747 5635 log.cc:1196] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/648655294c1c42339edcf74d72f4f4de
I20260807 20:08:16.073966 5635 ts_tablet_manager.cc:1939] T 648655294c1c42339edcf74d72f4f4de P 90edb56383614ba18f33dca3c81243f3: Deleting consensus metadata
I20260807 20:08:16.074493 4931 catalog_manager.cc:4841] TS 90edb56383614ba18f33dca3c81243f3 (127.3.15.130:34127): tablet 648655294c1c42339edcf74d72f4f4de (table test-table [id=a075388a10ca4b1996157a538f9060bb]) successfully deleted
W20260807 20:08:16.078235 4934 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:46013 (error 108)
I20260807 20:08:16.078498 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 5170
I20260807 20:08:16.079356 5639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:16.078696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46013 (local address 127.0.0.1:46820)
0807 20:08:16.079264 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:16.079317 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:46013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":67,"threads_started":1}
W20260807 20:08:16.079478 4934 catalog_manager.cc:4584] TS 2c3f0ae4a45d414cbcdfd396dc03c16b (127.3.15.129:46013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 648655294c1c42339edcf74d72f4f4de: Network error: Client connection negotiation failed: client connection to 127.3.15.129:46013: connect: Connection refused (error 111)
I20260807 20:08:16.086468 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 5318
I20260807 20:08:16.093195 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 4914
2026-08-07T20:08:16Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7666 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:16 dist-test-slave-1x4g krb5kdc[5642](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:16 dist-test-slave-1x4g krb5kdc[5642](info): set up 2 sockets
Aug 07 20:08:16 dist-test-slave-1x4g krb5kdc[5642](info): commencing operation
krb5kdc: starting...
W20260807 20:08:17.165807 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:17.295320 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44065
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:44065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:17.380671 5656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:17.380998 5656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:17.381044 5656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:17.382225 5656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:17.382300 5656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:17.382328 5656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:17.382360 5656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:17.382396 5656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:17.383915 5656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:44065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:17.384228 5656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:17.384481 5656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:17.390442 5662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:17.390480 5664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:17.390470 5661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.390514 5656 server_base.cc:1034] running on GCE node
I20260807 20:08:17.390820 5656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:17.391129 5656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:17.392302 5656 hybrid_clock.cc:648] HybridClock initialized: now 1786133297392296 us; error 36 us; skew 500 ppm
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:17.394805 5656 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:17.395287 5656 webserver.cc:460] Webserver started at http://127.3.15.190:41325/ using document root <none> and password file <none>
I20260807 20:08:17.395525 5656 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:17.395596 5656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:17.395750 5656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:17.396927 5656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "899ef1a4c3b94a9d91b0d07e962c759b"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "e82f27e9f665600a93845007824c0a71"
server_key_iv: "911f64fcfa06c77cb90dcf523939e801"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.397328 5656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "899ef1a4c3b94a9d91b0d07e962c759b"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "e82f27e9f665600a93845007824c0a71"
server_key_iv: "911f64fcfa06c77cb90dcf523939e801"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.398653 5656 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:17.399410 5671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.399600 5656 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:17.399708 5656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "899ef1a4c3b94a9d91b0d07e962c759b"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "e82f27e9f665600a93845007824c0a71"
server_key_iv: "911f64fcfa06c77cb90dcf523939e801"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.399780 5656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:17.411422 5656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:17.413560 5656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:17.413724 5656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:17.418130 5656 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:44065
I20260807 20:08:17.418162 5724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:44065 every 8 connection(s)
I20260807 20:08:17.418524 5656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:17.419409 5725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:17.420977 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 5656
I20260807 20:08:17.421079 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:17.421290 3134 external_mini_cluster.cc:1428] Setting key c2050dc3dc4f4a20b9ae7a2da866205b
I20260807 20:08:17.422169 5725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b: Bootstrap starting.
I20260807 20:08:17.422827 5725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:17.423182 5725 log.cc:824] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:17.423962 5725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b: No bootstrap required, opened a new log
I20260807 20:08:17.426200 5725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44065 } }
I20260807 20:08:17.426398 5725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:17.426445 5725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 899ef1a4c3b94a9d91b0d07e962c759b, State: Initialized, Role: FOLLOWER
I20260807 20:08:17.426576 5725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44065 } }
I20260807 20:08:17.426697 5725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:17.426743 5725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:17.426820 5725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:17.427773 5725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44065 } }
I20260807 20:08:17.427911 5725 leader_election.cc:302] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 899ef1a4c3b94a9d91b0d07e962c759b; no voters:
I20260807 20:08:17.428102 5725 leader_election.cc:288] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:17.428165 5731 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:17.428324 5731 raft_consensus.cc:686] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [term 1 LEADER]: Becoming Leader. State: Replica: 899ef1a4c3b94a9d91b0d07e962c759b, State: Running, Role: LEADER
I20260807 20:08:17.428408 5725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:17.428519 5731 consensus_queue.cc:237] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44065 } }
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:17.429037 5731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 899ef1a4c3b94a9d91b0d07e962c759b. Latest consensus state: current_term: 1 leader_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44065 } } }
I20260807 20:08:17.429046 5733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "899ef1a4c3b94a9d91b0d07e962c759b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44065 } } }
I20260807 20:08:17.429229 5731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:17.429236 5733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:17.429513 5740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:17.430397 5740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:17.433045 5740 catalog_manager.cc:1347] Generated new cluster ID: 17cc227f3bf449cca3f721e3327b9e26
I20260807 20:08:17.433128 5740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:17.433522 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.421857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.3.15.190:44065)
0807 20:08:17.422092 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:17.422100 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:17.422130 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:08:17.422635 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:17.422643 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:17.422699 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:17.422828 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:17.423662 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.424661 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.425805 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.426210 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.426308 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.429644 (+ 3336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:17.429649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:17.429649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:17.429661 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:17.431700 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.432165 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.432168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.432169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.432267 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.432496 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.432497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.432497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.432745 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:17.432982 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:17.433126 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 20:08:17.433282 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":99,"threads_started":1}
I20260807 20:08:17.448684 5740 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:17.449246 5740 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:17.454298 5740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 899ef1a4c3b94a9d91b0d07e962c759b: Generated new TSK 0
I20260807 20:08:17.454512 5740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:17.489919 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44065
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:17.574697 5754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:17.574903 5754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:17.574927 5754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:17.576094 5754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:17.576141 5754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:17.576167 5754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:17.577719 5754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:17.577940 5754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:17.578156 5754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:17.583482 5761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:17.583482 5759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.583509 5754 server_base.cc:1034] running on GCE node
W20260807 20:08:17.583484 5763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.584120 5754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:17.584369 5754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:17.585532 5754 hybrid_clock.cc:648] HybridClock initialized: now 1786133297585518 us; error 31 us; skew 500 ppm
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:17.588199 5754 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:17.588644 5754 webserver.cc:460] Webserver started at http://127.3.15.129:42489/ using document root <none> and password file <none>
I20260807 20:08:17.588922 5754 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:17.589011 5754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:17.589164 5754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:17.590214 5754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "29a2c9b125df4dcc83dcf799960960e5"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "06ec41f9023d332b72578fc912ee0835"
server_key_iv: "e6ac87695e90ecb5e7b7f274ec7e962a"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.590585 5754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "29a2c9b125df4dcc83dcf799960960e5"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "06ec41f9023d332b72578fc912ee0835"
server_key_iv: "e6ac87695e90ecb5e7b7f274ec7e962a"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.591933 5754 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:17.592645 5770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.592821 5754 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:17.592916 5754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "29a2c9b125df4dcc83dcf799960960e5"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "06ec41f9023d332b72578fc912ee0835"
server_key_iv: "e6ac87695e90ecb5e7b7f274ec7e962a"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.592973 5754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:17.609894 5754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:17.611697 5754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:17.611807 5754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:17.611990 5754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:17.612326 5754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:17.612383 5754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.612427 5754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:17.612468 5754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.618271 5754 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:35683
I20260807 20:08:17.618348 5891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:35683 every 8 connection(s)
I20260807 20:08:17.618687 5754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:17.626098 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 5754
I20260807 20:08:17.626202 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:17.626415 3134 external_mini_cluster.cc:1428] Setting key 2cc66bd328171901587da5e338c4221f
I20260807 20:08:17.628510 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.619442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:40097 (local address 127.3.15.190:44065)
0807 20:08:17.619511 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:17.619513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:17.619720 (+ 207us) server_negotiation.cc:407] Connection header received
0807 20:08:17.620273 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:17.620274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:17.620288 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:17.620340 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:17.621346 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.621873 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.622890 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.623124 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.623211 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.625355 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:17.625356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:17.625356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:17.625364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:17.626964 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.627397 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.627398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.627399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.627466 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.627709 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.627710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.627710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.627865 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:17.627946 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:17.628240 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 20:08:17.628344 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:17.628531 5894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.619405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44065 (local address 127.3.15.129:40097)
0807 20:08:17.619602 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:17.619638 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:17.620155 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:17.620415 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:17.620418 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:17.621042 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:17.621243 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:17.621249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.621976 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.621978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:17.622794 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:17.622796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.623229 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.623230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:17.623358 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.623818 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:17.623827 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:17.625246 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:17.627046 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:17.627055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:17.627057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:17.627318 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:17.627536 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:17.627537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:17.627537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:17.627643 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:17.627929 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:17.627934 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:17.628080 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 20:08:17.628199 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 20:08:17.628326 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260807 20:08:17.629351 5892 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44065
I20260807 20:08:17.629460 5892 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:17.629693 5892 heartbeater.cc:511] Master 127.3.15.190:44065 requested a full tablet report, sending...
I20260807 20:08:17.630333 5688 ts_manager.cc:194] Registered new tserver with Master: 29a2c9b125df4dcc83dcf799960960e5 (127.3.15.129:35683)
I20260807 20:08:17.630931 5688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:40097
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:17.656165 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44065
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:17.740726 5897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:17.740962 5897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:17.740986 5897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:17.742185 5897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:17.742229 5897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:17.742255 5897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:17.743623 5897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:17.743813 5897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:17.743989 5897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:17.748993 5903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.749166 5897 server_base.cc:1034] running on GCE node
W20260807 20:08:17.749037 5906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:17.748992 5904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.749464 5897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:17.749665 5897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:17.750802 5897 hybrid_clock.cc:648] HybridClock initialized: now 1786133297750770 us; error 48 us; skew 500 ppm
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:17.753566 5897 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:17.754051 5897 webserver.cc:460] Webserver started at http://127.3.15.130:36657/ using document root <none> and password file <none>
I20260807 20:08:17.754302 5897 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:17.754349 5897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:17.754511 5897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:17.755546 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "ac4adb5612ea43659c2017580d07bafc"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "5453cd7763320560189b2ef6ea1ab821"
server_key_iv: "83150951a95ec37207f9b3feabcf51ad"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.755920 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "ac4adb5612ea43659c2017580d07bafc"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "5453cd7763320560189b2ef6ea1ab821"
server_key_iv: "83150951a95ec37207f9b3feabcf51ad"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.757323 5897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:17.758104 5914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.758286 5897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:17.758396 5897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "ac4adb5612ea43659c2017580d07bafc"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "5453cd7763320560189b2ef6ea1ab821"
server_key_iv: "83150951a95ec37207f9b3feabcf51ad"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.758471 5897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:17.770006 5897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:17.772230 5897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:17.772378 5897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:17.772588 5897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:17.772958 5897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:17.773012 5897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.773046 5897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:17.773073 5897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.779132 5897 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:38771
I20260807 20:08:17.779191 6037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:38771 every 8 connection(s)
I20260807 20:08:17.779524 5897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:17.781903 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 5897
I20260807 20:08:17.782004 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:17.782202 3134 external_mini_cluster.cc:1428] Setting key 7e79e75d49182f4a32b104dcc030920b
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:17.790112 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.780336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46817 (local address 127.3.15.190:44065)
0807 20:08:17.780395 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:17.780397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:17.780575 (+ 178us) server_negotiation.cc:407] Connection header received
0807 20:08:17.781172 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:17.781173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:17.781185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:17.781251 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:17.782466 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.783072 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.784578 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.784851 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.784932 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.787153 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:17.787155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:17.787155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:17.787163 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:17.788588 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.789105 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.789107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.789107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.789173 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.789432 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.789433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.789433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.789557 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:17.789650 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:17.789934 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 20:08:17.790021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:08:17.790323 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.780282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44065 (local address 127.3.15.130:46817)
0807 20:08:17.780442 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:17.780485 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:17.781061 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:17.781307 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:17.781311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:17.782045 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:17.782340 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:17.782347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.783167 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.783169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:17.784486 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:17.784490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.784945 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.784946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:17.785076 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.785584 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:17.785592 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:17.787022 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:17.788704 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:17.788712 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:17.788714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:17.789012 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:17.789243 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:17.789244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:17.789244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:17.789362 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:17.789644 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:17.789652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:17.789794 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 20:08:17.789899 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 20:08:17.790063 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260807 20:08:17.790784 6038 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44065
I20260807 20:08:17.790884 6038 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:17.791093 6038 heartbeater.cc:511] Master 127.3.15.190:44065 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
I20260807 20:08:17.791518 5688 ts_manager.cc:194] Registered new tserver with Master: ac4adb5612ea43659c2017580d07bafc (127.3.15.130:38771)
I20260807 20:08:17.791908 5688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:46817
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:17.812114 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44065
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:17.896883 6043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:17.897084 6043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:17.897106 6043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:17.898252 6043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:17.898299 6043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:17.898348 6043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:17.899766 6043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:17.900010 6043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:17.900240 6043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:17.905247 6050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:17.905254 6049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.905308 6043 server_base.cc:1034] running on GCE node
W20260807 20:08:17.905234 6052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:17.905648 6043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:17.905903 6043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:17.907100 6043 hybrid_clock.cc:648] HybridClock initialized: now 1786133297907082 us; error 33 us; skew 500 ppm
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:17.909915 6043 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:17.910429 6043 webserver.cc:460] Webserver started at http://127.3.15.131:44695/ using document root <none> and password file <none>
I20260807 20:08:17.910681 6043 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:17.910728 6043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:17.910889 6043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:17.911980 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "73867738f9a34dff8ed0917ecead19a7"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "b05ca8113813a4c11febbcdd92723814"
server_key_iv: "96f083521269920325c2d985df7c8a34"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.912348 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "73867738f9a34dff8ed0917ecead19a7"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "b05ca8113813a4c11febbcdd92723814"
server_key_iv: "96f083521269920325c2d985df7c8a34"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.913777 6043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:17.914589 6060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.914784 6043 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:08:17.914896 6043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "73867738f9a34dff8ed0917ecead19a7"
format_stamp: "Formatted at 2026-08-07 20:08:17 on dist-test-slave-1x4g"
server_key: "b05ca8113813a4c11febbcdd92723814"
server_key_iv: "96f083521269920325c2d985df7c8a34"
server_key_version: "encryptionkey@0"
I20260807 20:08:17.915014 6043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:17.942368 6043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:17.944335 6043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:17.944454 6043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:17.944660 6043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:17.945021 6043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:17.945084 6043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.945127 6043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:17.945151 6043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:17.951187 6043 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:38595
I20260807 20:08:17.951233 6176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:38595 every 8 connection(s)
I20260807 20:08:17.951557 6043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:08:17 dist-test-slave-1x4g krb5kdc[5642](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133297, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:17.958503 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6043
I20260807 20:08:17.958633 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:17.958814 3134 external_mini_cluster.cc:1428] Setting key 9a76823b12398eeb35c196f7b858123e
I20260807 20:08:17.961155 5728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.952250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:57937 (local address 127.3.15.190:44065)
0807 20:08:17.952317 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:17.952319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:17.952512 (+ 193us) server_negotiation.cc:407] Connection header received
0807 20:08:17.953107 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:17.953108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:17.953118 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:17.953177 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:17.954139 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.954848 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.955837 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.956017 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.956067 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.958265 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:17.958267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:17.958267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:17.958274 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:17.959918 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.960299 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.960300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.960300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.960352 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:17.960567 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:17.960568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:17.960568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:17.960671 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:17.960722 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:17.961027 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 20:08:17.961091 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:08:17.961407 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:17.952204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44065 (local address 127.3.15.131:57937)
0807 20:08:17.952368 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:17.952411 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:17.953010 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:17.953247 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:17.953254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:17.953839 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:17.954041 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:17.954046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.954946 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.954947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:17.955763 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:17.955765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:17.956092 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:17.956093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:17.956200 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:17.956656 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:17.956665 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:17.958176 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:17.959996 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:17.960002 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:17.960004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:17.960230 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:17.960410 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:17.960411 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:17.960411 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:17.960511 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:17.960727 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:17.960731 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:17.960907 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 20:08:17.960995 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:08:17.961129 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260807 20:08:17.961900 6177 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44065
I20260807 20:08:17.962006 6177 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:17.962196 6177 heartbeater.cc:511] Master 127.3.15.190:44065 requested a full tablet report, sending...
I20260807 20:08:17.962616 5688 ts_manager.cc:194] Registered new tserver with Master: 73867738f9a34dff8ed0917ecead19a7 (127.3.15.131:38595)
I20260807 20:08:17.962984 5688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:57937
I20260807 20:08:17.970461 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 20:08:17.984081 5728 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:17.983081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38458 (local address 127.3.15.190:44065)
0807 20:08:17.983170 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:17.983172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:17.983229 (+ 57us) server_negotiation.cc:407] Connection header received
0807 20:08:17.983289 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:17.983290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:17.983299 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:17.983350 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:17.983992 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38458: BlockingRecv error: recv got EOF from 127.0.0.1:38458 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 20:08:17.984948 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 5754
I20260807 20:08:17.991466 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 5897
I20260807 20:08:17.997211 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6043
I20260807 20:08:18.001940 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 5656
2026-08-07T20:08:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1907 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:18 dist-test-slave-1x4g krb5kdc[6191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:18 dist-test-slave-1x4g krb5kdc[6191](info): set up 2 sockets
Aug 07 20:08:18 dist-test-slave-1x4g krb5kdc[6191](info): commencing operation
krb5kdc: starting...
W20260807 20:08:19.069430 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:19 dist-test-slave-1x4g krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:19.192317 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:41255
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:41255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:19.278474 6206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:19.278689 6206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:19.278712 6206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:19.279852 6206 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:19.279891 6206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:19.279906 6206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:19.279918 6206 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:19.279934 6206 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:19.281428 6206 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:41255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:41255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:19.281665 6206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:19.281899 6206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:19.287165 6212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:19.287189 6211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:19.287520 6206 server_base.cc:1034] running on GCE node
W20260807 20:08:19.287169 6214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:19.287894 6206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:19.288170 6206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:19.289312 6206 hybrid_clock.cc:648] HybridClock initialized: now 1786133299289301 us; error 30 us; skew 500 ppm
Aug 07 20:08:19 dist-test-slave-1x4g krb5kdc[6191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133299, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:19.292109 6206 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:19.292750 6206 webserver.cc:460] Webserver started at http://127.3.15.190:35351/ using document root <none> and password file <none>
I20260807 20:08:19.293022 6206 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:19.293073 6206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:19.293170 6206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:19.294247 6206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "f0878192fde947eaa348410c44397fa3"
format_stamp: "Formatted at 2026-08-07 20:08:19 on dist-test-slave-1x4g"
server_key: "df1230a3a0bf559ca08d98d8ab56239b"
server_key_iv: "837519c0bc53bfa271ee9a00b67c3730"
server_key_version: "encryptionkey@0"
I20260807 20:08:19.294569 6206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "f0878192fde947eaa348410c44397fa3"
format_stamp: "Formatted at 2026-08-07 20:08:19 on dist-test-slave-1x4g"
server_key: "df1230a3a0bf559ca08d98d8ab56239b"
server_key_iv: "837519c0bc53bfa271ee9a00b67c3730"
server_key_version: "encryptionkey@0"
I20260807 20:08:19.295877 6206 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:19.296607 6221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:19.296859 6206 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:19.296939 6206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "f0878192fde947eaa348410c44397fa3"
format_stamp: "Formatted at 2026-08-07 20:08:19 on dist-test-slave-1x4g"
server_key: "df1230a3a0bf559ca08d98d8ab56239b"
server_key_iv: "837519c0bc53bfa271ee9a00b67c3730"
server_key_version: "encryptionkey@0"
I20260807 20:08:19.296989 6206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:19.314678 6206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:19.316967 6206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:19.317090 6206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:19.321794 6206 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:41255
I20260807 20:08:19.321852 6274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:41255 every 8 connection(s)
I20260807 20:08:19.322211 6206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:19.323117 6275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:19.325551 6275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3: Bootstrap starting.
I20260807 20:08:19.326180 6275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:19.326502 6275 log.cc:824] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:19.327243 6275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3: No bootstrap required, opened a new log
I20260807 20:08:19.327778 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6206
I20260807 20:08:19.327884 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:19.328088 3134 external_mini_cluster.cc:1428] Setting key f5381a898a957fb68aa7b2f2817c09b1
I20260807 20:08:19.329981 6275 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0878192fde947eaa348410c44397fa3" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41255 } }
I20260807 20:08:19.330163 6275 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:19.330207 6275 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0878192fde947eaa348410c44397fa3, State: Initialized, Role: FOLLOWER
I20260807 20:08:19.330354 6275 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0878192fde947eaa348410c44397fa3" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41255 } }
I20260807 20:08:19.330449 6275 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:19.330500 6275 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:19.330556 6275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:19.331283 6275 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0878192fde947eaa348410c44397fa3" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41255 } }
I20260807 20:08:19.331378 6275 leader_election.cc:302] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0878192fde947eaa348410c44397fa3; no voters:
I20260807 20:08:19.331604 6275 leader_election.cc:288] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:19.331640 6281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:19.331746 6281 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [term 1 LEADER]: Becoming Leader. State: Replica: f0878192fde947eaa348410c44397fa3, State: Running, Role: LEADER
I20260807 20:08:19.331835 6281 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0878192fde947eaa348410c44397fa3" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41255 } }
I20260807 20:08:19.331892 6275 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:19.332211 6281 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0878192fde947eaa348410c44397fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0878192fde947eaa348410c44397fa3" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41255 } } }
I20260807 20:08:19.332301 6281 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:19.332432 6281 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0878192fde947eaa348410c44397fa3. Latest consensus state: current_term: 1 leader_uuid: "f0878192fde947eaa348410c44397fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0878192fde947eaa348410c44397fa3" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41255 } } }
I20260807 20:08:19.332496 6281 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:19.332649 6286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:19.333242 6286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:19.334991 6286 catalog_manager.cc:1347] Generated new cluster ID: 6be8fa6a51374fe89c4b3cb131b9730a
I20260807 20:08:19.335055 6286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:19 dist-test-slave-1x4g krb5kdc[6191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:19.341356 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:19.328618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.3.15.190:41255)
0807 20:08:19.328831 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:19.328843 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:19.328875 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:08:19.329964 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:19.329972 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:19.330027 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:19.330199 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:19.331052 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:19.332331 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:19.333549 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:19.334142 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:19.334264 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:19.337282 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:19.337287 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:19.337288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:19.337302 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:19.339388 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:19.339825 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:19.339829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:19.339829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:19.339962 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:19.340248 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:19.340249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:19.340249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:19.340625 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:19.340815 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:19.340931 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:08:19.341088 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260807 20:08:19.342324 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:08:19.350598 6286 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:19.351398 6286 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:19.355000 6286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0878192fde947eaa348410c44397fa3: Generated new TSK 0
I20260807 20:08:19.355183 6286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:19.361356 6280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:19.356245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48194 (local address 127.3.15.190:41255)
0807 20:08:19.356307 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:19.356309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:19.356984 (+ 675us) server_negotiation.cc:407] Connection header received
0807 20:08:19.357023 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:19.357024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:19.357036 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:19.357106 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:19.357503 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:19.358880 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:19.359648 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:19.360006 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:19.360074 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:19.360845 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:19.360846 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:19.360846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:19.361070 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:19.361090 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:19.361098 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:19.361162 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:19.361246 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 20:08:19.361275 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:08:19.364459 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6206
2026-08-07T20:08:19Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1360 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:19 dist-test-slave-1x4g krb5kdc[6310](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:19 dist-test-slave-1x4g krb5kdc[6310](info): set up 2 sockets
Aug 07 20:08:19 dist-test-slave-1x4g krb5kdc[6310](info): commencing operation
krb5kdc: starting...
W20260807 20:08:20.481015 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:20 dist-test-slave-1x4g krb5kdc[6310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:20.620105 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:37419
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:20.704913 6323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:20.705174 6323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:20.705214 6323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:20.706398 6323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:20.706471 6323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:20.706496 6323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:20.706516 6323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:20.706547 6323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:20.708101 6323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:20.708405 6323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:20.708657 6323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:20.713685 6329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:20.713739 6331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:20.713830 6323 server_base.cc:1034] running on GCE node
W20260807 20:08:20.713685 6328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:20.714223 6323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:20.714526 6323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:20.715693 6323 hybrid_clock.cc:648] HybridClock initialized: now 1786133300715680 us; error 48 us; skew 500 ppm
Aug 07 20:08:20 dist-test-slave-1x4g krb5kdc[6310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133300, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:20.718400 6323 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:20.718883 6323 webserver.cc:460] Webserver started at http://127.3.15.190:42821/ using document root <none> and password file <none>
I20260807 20:08:20.719141 6323 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:20.719211 6323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:20.719360 6323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:20.720450 6323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "f11dfe138b5549f2b88ae60880c18df6"
format_stamp: "Formatted at 2026-08-07 20:08:20 on dist-test-slave-1x4g"
server_key: "076bc995574f7d6ff87db625f20f1402"
server_key_iv: "1b7d9d597dae964cc5c384fe6952efbc"
server_key_version: "encryptionkey@0"
I20260807 20:08:20.720887 6323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "f11dfe138b5549f2b88ae60880c18df6"
format_stamp: "Formatted at 2026-08-07 20:08:20 on dist-test-slave-1x4g"
server_key: "076bc995574f7d6ff87db625f20f1402"
server_key_iv: "1b7d9d597dae964cc5c384fe6952efbc"
server_key_version: "encryptionkey@0"
I20260807 20:08:20.722227 6323 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:20.723016 6338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:20.723206 6323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:08:20.723310 6323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "f11dfe138b5549f2b88ae60880c18df6"
format_stamp: "Formatted at 2026-08-07 20:08:20 on dist-test-slave-1x4g"
server_key: "076bc995574f7d6ff87db625f20f1402"
server_key_iv: "1b7d9d597dae964cc5c384fe6952efbc"
server_key_version: "encryptionkey@0"
I20260807 20:08:20.723392 6323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:20.744890 6323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:20.747027 6323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:20.747200 6323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:20.751541 6323 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:37419
I20260807 20:08:20.751602 6393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:37419 every 8 connection(s)
I20260807 20:08:20.751981 6323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:20.753261 6394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:20.755604 6394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6: Bootstrap starting.
I20260807 20:08:20.755770 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6323
I20260807 20:08:20.755894 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:20.756078 3134 external_mini_cluster.cc:1428] Setting key 2d41e3bf7d655745d2579c0fd8253e28
I20260807 20:08:20.756337 6394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:20.756960 6394 log.cc:824] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:20.757733 6394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6: No bootstrap required, opened a new log
I20260807 20:08:20.760119 6394 raft_consensus.cc:348] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11dfe138b5549f2b88ae60880c18df6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37419 } }
I20260807 20:08:20.760262 6394 raft_consensus.cc:374] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:20.760284 6394 raft_consensus.cc:729] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f11dfe138b5549f2b88ae60880c18df6, State: Initialized, Role: FOLLOWER
I20260807 20:08:20.760419 6394 consensus_queue.cc:260] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11dfe138b5549f2b88ae60880c18df6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37419 } }
I20260807 20:08:20.760515 6394 raft_consensus.cc:388] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:20.760560 6394 raft_consensus.cc:482] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:20.760612 6394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:20.761353 6394 raft_consensus.cc:504] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11dfe138b5549f2b88ae60880c18df6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37419 } }
I20260807 20:08:20.761471 6394 leader_election.cc:302] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f11dfe138b5549f2b88ae60880c18df6; no voters:
I20260807 20:08:20.761644 6394 leader_election.cc:288] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:20.761713 6399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:20.761909 6399 raft_consensus.cc:686] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [term 1 LEADER]: Becoming Leader. State: Replica: f11dfe138b5549f2b88ae60880c18df6, State: Running, Role: LEADER
I20260807 20:08:20.761933 6394 sys_catalog.cc:564] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:20.762006 6399 consensus_queue.cc:237] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11dfe138b5549f2b88ae60880c18df6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37419 } }
I20260807 20:08:20.762370 6399 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f11dfe138b5549f2b88ae60880c18df6. Latest consensus state: current_term: 1 leader_uuid: "f11dfe138b5549f2b88ae60880c18df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11dfe138b5549f2b88ae60880c18df6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37419 } } }
I20260807 20:08:20.762544 6400 sys_catalog.cc:455] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f11dfe138b5549f2b88ae60880c18df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f11dfe138b5549f2b88ae60880c18df6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37419 } } }
I20260807 20:08:20.762589 6399 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:20.762691 6400 sys_catalog.cc:458] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:20.762923 6408 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:20.763494 6408 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 20:08:20 dist-test-slave-1x4g krb5kdc[6310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:20.765197 6408 catalog_manager.cc:1347] Generated new cluster ID: ea69efda9d3f4847a25ea50af700ecdf
I20260807 20:08:20.765251 6408 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:20.769479 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:20.756675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54976 (local address 127.3.15.190:37419)
0807 20:08:20.756850 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:20.756861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:20.756899 (+ 38us) server_negotiation.cc:407] Connection header received
0807 20:08:20.757506 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:20.757516 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:20.757601 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:20.757760 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:20.758423 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:20.759916 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:20.761088 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:20.761735 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:20.762229 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:20.765193 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:20.765197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:20.765197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:20.765211 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:20.767279 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:20.767706 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:20.767709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:20.767709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:20.767814 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:20.768041 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:20.768042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:20.768043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:20.768369 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:20.769014 (+ 645us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:20.769104 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 20:08:20.769252 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":86,"threads_started":1}
I20260807 20:08:20.776499 6408 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:20.777182 6408 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:20.782048 6408 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f11dfe138b5549f2b88ae60880c18df6: Generated new TSK 0
I20260807 20:08:20.782234 6408 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:20.816182 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37419
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:20.904363 6420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:20.904618 6420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:20.904657 6420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:20.905864 6420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:20.905959 6420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:20.906025 6420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:20.907538 6420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:20.907823 6420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:20.908085 6420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:20.913466 6427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:20.913504 6429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:20.913522 6425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:20.913682 6420 server_base.cc:1034] running on GCE node
I20260807 20:08:20.914006 6420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:20.914276 6420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:20.915464 6420 hybrid_clock.cc:648] HybridClock initialized: now 1786133300915460 us; error 63 us; skew 500 ppm
Aug 07 20:08:20 dist-test-slave-1x4g krb5kdc[6310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133300, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:20.918159 6420 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:20.918610 6420 webserver.cc:460] Webserver started at http://127.3.15.129:35087/ using document root <none> and password file <none>
I20260807 20:08:20.918859 6420 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:20.918931 6420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:20.919083 6420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:20.920166 6420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "f3eb1747efb84bb188020f9ad900fc7f"
format_stamp: "Formatted at 2026-08-07 20:08:20 on dist-test-slave-1x4g"
server_key: "449e0a6ab1d100b1f5f64a1ad108df64"
server_key_iv: "0cf74005bfa899b2952d83720bb03d61"
server_key_version: "encryptionkey@0"
I20260807 20:08:20.920538 6420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "f3eb1747efb84bb188020f9ad900fc7f"
format_stamp: "Formatted at 2026-08-07 20:08:20 on dist-test-slave-1x4g"
server_key: "449e0a6ab1d100b1f5f64a1ad108df64"
server_key_iv: "0cf74005bfa899b2952d83720bb03d61"
server_key_version: "encryptionkey@0"
I20260807 20:08:20.922091 6420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260807 20:08:20.922909 6436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:20.923226 6420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:20.923336 6420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "f3eb1747efb84bb188020f9ad900fc7f"
format_stamp: "Formatted at 2026-08-07 20:08:20 on dist-test-slave-1x4g"
server_key: "449e0a6ab1d100b1f5f64a1ad108df64"
server_key_iv: "0cf74005bfa899b2952d83720bb03d61"
server_key_version: "encryptionkey@0"
I20260807 20:08:20.923413 6420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:20.937712 6420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:20.939620 6420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:20.939786 6420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:20.940016 6420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:20.940330 6420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:20.940384 6420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:20.940418 6420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:20.940445 6420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:20.946321 6420 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:39339
I20260807 20:08:20.946408 6558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:39339 every 8 connection(s)
I20260807 20:08:20.946753 6420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:20.952200 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6420
I20260807 20:08:20.952282 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:20.952519 3134 external_mini_cluster.cc:1428] Setting key 6eb420409bfb2a9bdfdc6030fb22f54e
Aug 07 20:08:20 dist-test-slave-1x4g krb5kdc[6310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133300, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:20.956743 6562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:20.947503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37419 (local address 127.3.15.129:35063)
0807 20:08:20.947682 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:20.947719 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:20.948209 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:20.948418 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:20.948421 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:20.949052 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:20.949253 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:20.949259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:20.949918 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:20.949918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:20.950751 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:20.950753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:20.951103 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:20.951104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:20.951204 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:20.951759 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:20.951768 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:20.953484 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:20.955401 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:20.955407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:20.955409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:20.955625 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:20.955839 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:20.955840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:20.955840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:20.955940 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:20.956247 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:20.956252 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:20.956367 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 20:08:20.956449 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 20:08:20.956568 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":111,"threads_started":1}
I20260807 20:08:20.956743 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:20.947563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:35063 (local address 127.3.15.190:37419)
0807 20:08:20.947654 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:20.947656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:20.947811 (+ 155us) server_negotiation.cc:407] Connection header received
0807 20:08:20.948296 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:20.948297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:20.948307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:20.948355 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:20.949336 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:20.949835 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:20.950827 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:20.951028 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:20.951079 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:20.953574 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:20.953575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:20.953575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:20.953582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:20.955315 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:20.955700 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:20.955701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:20.955702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:20.955770 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:20.956011 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:20.956012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:20.956012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:20.956179 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:20.956254 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:20.956497 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 20:08:20.956594 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 20:08:20.957409 6559 heartbeater.cc:347] Connected to a master server at 127.3.15.190:37419
I20260807 20:08:20.957533 6559 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:20.957717 6559 heartbeater.cc:511] Master 127.3.15.190:37419 requested a full tablet report, sending...
I20260807 20:08:20.958350 6356 ts_manager.cc:194] Registered new tserver with Master: f3eb1747efb84bb188020f9ad900fc7f (127.3.15.129:39339)
I20260807 20:08:20.958964 6356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:35063
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:20.979660 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37419
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:21.062984 6565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:21.063215 6565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:21.063254 6565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:21.064502 6565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:21.064575 6565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:21.064626 6565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:21.066174 6565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:21.066454 6565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:21.066708 6565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:21.072368 6570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:21.072398 6574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:21.072414 6571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:21.072610 6565 server_base.cc:1034] running on GCE node
I20260807 20:08:21.072870 6565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:21.073163 6565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:21.074337 6565 hybrid_clock.cc:648] HybridClock initialized: now 1786133301074324 us; error 25 us; skew 500 ppm
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133301, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:21.077122 6565 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:21.077618 6565 webserver.cc:460] Webserver started at http://127.3.15.130:34443/ using document root <none> and password file <none>
I20260807 20:08:21.077873 6565 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:21.077943 6565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:21.078105 6565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:21.079234 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "857ebdead7a24a5bb0e2c39fd724c870"
format_stamp: "Formatted at 2026-08-07 20:08:21 on dist-test-slave-1x4g"
server_key: "d0b7603471ad3bf3d7a52184227fd121"
server_key_iv: "683ec12c30c842b89303ad183d118288"
server_key_version: "encryptionkey@0"
I20260807 20:08:21.079648 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "857ebdead7a24a5bb0e2c39fd724c870"
format_stamp: "Formatted at 2026-08-07 20:08:21 on dist-test-slave-1x4g"
server_key: "d0b7603471ad3bf3d7a52184227fd121"
server_key_iv: "683ec12c30c842b89303ad183d118288"
server_key_version: "encryptionkey@0"
I20260807 20:08:21.081058 6565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:21.081789 6582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:21.081957 6565 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:21.082062 6565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "857ebdead7a24a5bb0e2c39fd724c870"
format_stamp: "Formatted at 2026-08-07 20:08:21 on dist-test-slave-1x4g"
server_key: "d0b7603471ad3bf3d7a52184227fd121"
server_key_iv: "683ec12c30c842b89303ad183d118288"
server_key_version: "encryptionkey@0"
I20260807 20:08:21.082136 6565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:21.105204 6565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:21.107689 6565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:21.107869 6565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:21.108114 6565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:21.108455 6565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:21.108510 6565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:21.108543 6565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:21.108569 6565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:21.114696 6565 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:46113
I20260807 20:08:21.114745 6704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:46113 every 8 connection(s)
I20260807 20:08:21.115069 6565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:21.115372 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6565
I20260807 20:08:21.115478 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:21.115701 3134 external_mini_cluster.cc:1428] Setting key fa9d4a1e5b8711d9fd8f0bae0855fb0b
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133301, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
I20260807 20:08:21.125581 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:21.115874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:35217 (local address 127.3.15.190:37419)
0807 20:08:21.115998 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:21.116001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:21.116150 (+ 149us) server_negotiation.cc:407] Connection header received
0807 20:08:21.116804 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:21.116805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:21.116818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:21.116863 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:21.117836 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.118349 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.119444 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.119659 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.119747 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:21.122439 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:21.122441 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:21.122441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:21.122448 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:21.123951 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:21.124475 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:21.124476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:21.124476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:21.124533 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:21.124891 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:21.124892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:21.124892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:21.125030 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:21.125088 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:21.125374 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 20:08:21.125466 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 20:08:21.125802 6708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:21.115833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37419 (local address 127.3.15.130:35217)
0807 20:08:21.116008 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:21.116059 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:21.116675 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:21.116942 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:21.116946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:21.117530 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:21.117728 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:21.117733 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.118472 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.118473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:21.119336 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:21.119338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.119760 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.119762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:21.119890 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:21.120422 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:21.120436 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:21.122338 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:21.124041 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:21.124050 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:21.124052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:21.124389 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:21.124606 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:21.124607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:21.124607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:21.124759 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:21.125116 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:21.125127 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:21.125254 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 20:08:21.125344 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:08:21.125514 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260807 20:08:21.126350 6705 heartbeater.cc:347] Connected to a master server at 127.3.15.190:37419
I20260807 20:08:21.126457 6705 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:21.126609 6705 heartbeater.cc:511] Master 127.3.15.190:37419 requested a full tablet report, sending...
I20260807 20:08:21.127161 6356 ts_manager.cc:194] Registered new tserver with Master: 857ebdead7a24a5bb0e2c39fd724c870 (127.3.15.130:46113)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
I20260807 20:08:21.127681 6356 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:35217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:21.144361 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37419
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:40597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:21.228281 6711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:21.228493 6711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:21.228516 6711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:21.229732 6711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:21.229774 6711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:21.229801 6711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:21.231242 6711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:21.231441 6711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:21.231633 6711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:21.236729 6719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:21.236726 6716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:21.236706 6717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:21.236877 6711 server_base.cc:1034] running on GCE node
I20260807 20:08:21.237074 6711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:21.237282 6711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:21.238448 6711 hybrid_clock.cc:648] HybridClock initialized: now 1786133301238423 us; error 39 us; skew 500 ppm
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133301, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:21.246095 6711 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:21.246558 6711 webserver.cc:460] Webserver started at http://127.3.15.131:44287/ using document root <none> and password file <none>
I20260807 20:08:21.246799 6711 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:21.246866 6711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:21.247025 6711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:21.248126 6711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "44e447e57a8a49c9b9cdbf76bff74034"
format_stamp: "Formatted at 2026-08-07 20:08:21 on dist-test-slave-1x4g"
server_key: "8e6f97247622520f69a474fe81d97470"
server_key_iv: "89465b0b147fa1b104d378654ad09832"
server_key_version: "encryptionkey@0"
I20260807 20:08:21.248566 6711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "44e447e57a8a49c9b9cdbf76bff74034"
format_stamp: "Formatted at 2026-08-07 20:08:21 on dist-test-slave-1x4g"
server_key: "8e6f97247622520f69a474fe81d97470"
server_key_iv: "89465b0b147fa1b104d378654ad09832"
server_key_version: "encryptionkey@0"
I20260807 20:08:21.250033 6711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:21.250849 6727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:21.251060 6711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:21.251163 6711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "44e447e57a8a49c9b9cdbf76bff74034"
format_stamp: "Formatted at 2026-08-07 20:08:21 on dist-test-slave-1x4g"
server_key: "8e6f97247622520f69a474fe81d97470"
server_key_iv: "89465b0b147fa1b104d378654ad09832"
server_key_version: "encryptionkey@0"
I20260807 20:08:21.251242 6711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:21.261730 6711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:21.264065 6711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:21.264227 6711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:21.264437 6711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:21.264740 6711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:21.264818 6711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:21.264869 6711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:21.264884 6711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:21.270790 6711 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:42147
I20260807 20:08:21.270819 6856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:42147 every 8 connection(s)
I20260807 20:08:21.271183 6711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133301, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:21.280201 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6711
I20260807 20:08:21.280297 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:21.280346 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:21.271943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:47883 (local address 127.3.15.190:37419)
0807 20:08:21.271998 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:21.272000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:21.272198 (+ 198us) server_negotiation.cc:407] Connection header received
0807 20:08:21.272768 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:21.272769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:21.272801 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:21.272853 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:21.273833 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.274334 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.275339 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.275543 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.275630 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:21.277791 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:21.277793 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:21.277793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:21.277801 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:21.279011 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:21.279415 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:21.279416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:21.279416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:21.279466 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:21.279688 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:21.279689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:21.279689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:21.279789 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:21.279840 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:21.280165 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 20:08:21.280228 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:08:21.280511 3134 external_mini_cluster.cc:1428] Setting key a445bd0e5c087825438e5ed4abf35e5a
I20260807 20:08:21.280599 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:21.271922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37419 (local address 127.3.15.131:47883)
0807 20:08:21.272084 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:21.272119 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:21.272652 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:21.272930 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:21.272933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:21.273529 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:21.273725 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:21.273730 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.274436 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.274437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:21.275251 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:21.275253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.275640 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.275641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:21.275767 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:21.276272 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:21.276281 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:21.277701 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:21.279105 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:21.279112 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:21.279114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:21.279344 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:21.279528 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:21.279529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:21.279529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:21.279632 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:21.279844 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:21.279849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:21.279974 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 20:08:21.280148 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 20:08:21.280333 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260807 20:08:21.281104 6857 heartbeater.cc:347] Connected to a master server at 127.3.15.190:37419
I20260807 20:08:21.281213 6857 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:21.281420 6857 heartbeater.cc:511] Master 127.3.15.190:37419 requested a full tablet report, sending...
I20260807 20:08:21.281881 6355 ts_manager.cc:194] Registered new tserver with Master: 44e447e57a8a49c9b9cdbf76bff74034 (127.3.15.131:42147)
I20260807 20:08:21.282305 6355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:47883
I20260807 20:08:21.292362 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6310](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133301, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6310](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133301, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:21.317047 6397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:21.309952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54980 (local address 127.3.15.190:37419)
0807 20:08:21.310016 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:21.310018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:21.310128 (+ 110us) server_negotiation.cc:407] Connection header received
0807 20:08:21.310181 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:21.310182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:21.310193 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:21.310253 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:21.310823 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.311327 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.312040 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:21.312220 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:21.312285 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:21.314503 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:21.314505 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:21.314505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:21.314512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:21.315985 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:21.316282 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:21.316283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:21.316283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:21.316336 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:21.316496 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:21.316496 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:21.316497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:21.316657 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:21.316740 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:21.316875 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 20:08:21.316939 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 20:08:21.317332 6355 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54980
I20260807 20:08:21.317827 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6420
I20260807 20:08:21.323891 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6565
I20260807 20:08:21.329298 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6711
I20260807 20:08:21.335410 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6323
2026-08-07T20:08:21Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1971 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6870](info): set up 2 sockets
Aug 07 20:08:21 dist-test-slave-1x4g krb5kdc[6870](info): commencing operation
krb5kdc: starting...
W20260807 20:08:22.401930 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:22 dist-test-slave-1x4g krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:22.524091 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35937
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:36433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:35937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:22.607146 6884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:22.607390 6884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:22.607430 6884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:22.608584 6884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:22.608654 6884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:22.608677 6884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:22.608698 6884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:22.608728 6884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:22.610273 6884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:35937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:22.610561 6884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:22.610791 6884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:22.615741 6890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:22.615760 6892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:22.615756 6889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:22.616083 6884 server_base.cc:1034] running on GCE node
I20260807 20:08:22.616357 6884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:22.616695 6884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:22.617853 6884 hybrid_clock.cc:648] HybridClock initialized: now 1786133302617843 us; error 30 us; skew 500 ppm
Aug 07 20:08:22 dist-test-slave-1x4g krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:22.620316 6884 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:22.620870 6884 webserver.cc:460] Webserver started at http://127.3.15.190:45111/ using document root <none> and password file <none>
I20260807 20:08:22.621111 6884 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:22.621181 6884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:22.621315 6884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:22.622387 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "236f20bceb0c457f89775f8b183738da"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "5aa759d3a067b9eeea2d317fd65b413d"
server_key_iv: "53aeb58408c0fd2b5e8d83d7829139b3"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.622767 6884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "236f20bceb0c457f89775f8b183738da"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "5aa759d3a067b9eeea2d317fd65b413d"
server_key_iv: "53aeb58408c0fd2b5e8d83d7829139b3"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.624106 6884 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:22.624859 6901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:22.625031 6884 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:22.625133 6884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "236f20bceb0c457f89775f8b183738da"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "5aa759d3a067b9eeea2d317fd65b413d"
server_key_iv: "53aeb58408c0fd2b5e8d83d7829139b3"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.625214 6884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:22.646241 6884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:22.648262 6884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:22.648437 6884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:22.652881 6884 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:35937
I20260807 20:08:22.652921 6956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:35937 every 8 connection(s)
I20260807 20:08:22.653486 6884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:22.654250 6957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:22.656857 6957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da: Bootstrap starting.
I20260807 20:08:22.657522 6957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:22.657872 6957 log.cc:824] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:22.658633 6957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da: No bootstrap required, opened a new log
I20260807 20:08:22.659586 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6884
I20260807 20:08:22.659682 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:22.659868 3134 external_mini_cluster.cc:1428] Setting key 708d73f98a4d93c4c0071b55fc716b17
I20260807 20:08:22.661494 6957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236f20bceb0c457f89775f8b183738da" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35937 } }
I20260807 20:08:22.661685 6957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:22.661741 6957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236f20bceb0c457f89775f8b183738da, State: Initialized, Role: FOLLOWER
I20260807 20:08:22.661885 6957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236f20bceb0c457f89775f8b183738da" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35937 } }
I20260807 20:08:22.661971 6957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:22.661998 6957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:22.662078 6957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:22.662813 6957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236f20bceb0c457f89775f8b183738da" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35937 } }
I20260807 20:08:22.662943 6957 leader_election.cc:302] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 236f20bceb0c457f89775f8b183738da; no voters:
I20260807 20:08:22.663116 6957 leader_election.cc:288] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:22.663194 6962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:22.663336 6962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [term 1 LEADER]: Becoming Leader. State: Replica: 236f20bceb0c457f89775f8b183738da, State: Running, Role: LEADER
I20260807 20:08:22.663367 6957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:22.663445 6962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236f20bceb0c457f89775f8b183738da" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35937 } }
I20260807 20:08:22.663790 6964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "236f20bceb0c457f89775f8b183738da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236f20bceb0c457f89775f8b183738da" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35937 } } }
I20260807 20:08:22.663771 6962 sys_catalog.cc:455] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 236f20bceb0c457f89775f8b183738da. Latest consensus state: current_term: 1 leader_uuid: "236f20bceb0c457f89775f8b183738da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "236f20bceb0c457f89775f8b183738da" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35937 } } }
I20260807 20:08:22.663882 6964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:22.663887 6962 sys_catalog.cc:458] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:22.664135 6969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:22.664654 6969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:22.666723 6969 catalog_manager.cc:1347] Generated new cluster ID: 792dfcf960ae43949821432c80309a60
I20260807 20:08:22.666786 6969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:22 dist-test-slave-1x4g krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:22.673079 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:22.660461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40386 (local address 127.3.15.190:35937)
0807 20:08:22.660692 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:22.660704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:22.660746 (+ 42us) server_negotiation.cc:407] Connection header received
0807 20:08:22.661451 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:22.661461 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:22.661529 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:22.661693 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:22.662481 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:22.663913 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:22.665127 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:22.665588 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:22.665729 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:22.668844 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:22.668849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:22.668850 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:22.668863 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:22.670957 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:22.671377 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:22.671380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:22.671380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:22.671502 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:22.671703 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:22.671704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:22.671704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:22.671989 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:22.672144 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:22.672664 (+ 520us) server_negotiation.cc:299] Negotiation successful
0807 20:08:22.672863 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":91,"threads_started":1}
I20260807 20:08:22.684516 6969 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:22.685287 6969 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:22.692315 6969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 236f20bceb0c457f89775f8b183738da: Generated new TSK 0
I20260807 20:08:22.692541 6969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:22.723152 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35937
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:36433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:22.821677 6983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:22.821879 6983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:22.821909 6983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:22.823040 6983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:22.823081 6983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:22.823105 6983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:22.824626 6983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:22.824913 6983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:22.825145 6983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:22.831444 6992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:22.831414 6990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:22.831420 6994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:22.831926 6983 server_base.cc:1034] running on GCE node
I20260807 20:08:22.832180 6983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:22.832434 6983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:22.833606 6983 hybrid_clock.cc:648] HybridClock initialized: now 1786133302833589 us; error 39 us; skew 500 ppm
Aug 07 20:08:22 dist-test-slave-1x4g krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:22.837044 6983 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:22.837527 6983 webserver.cc:460] Webserver started at http://127.3.15.129:44003/ using document root <none> and password file <none>
I20260807 20:08:22.837759 6983 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:22.837803 6983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:22.837961 6983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:22.839035 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "78cf6b45f8784d23a0eb67263c040476"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "a81fe0cf5b8d269e1ea9156661847cbe"
server_key_iv: "d8fe1ce1b09299f6a03dd4c869e2beec"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.839406 6983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "78cf6b45f8784d23a0eb67263c040476"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "a81fe0cf5b8d269e1ea9156661847cbe"
server_key_iv: "d8fe1ce1b09299f6a03dd4c869e2beec"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.840718 6983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:22.841476 7001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:22.841660 6983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:22.841749 6983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "78cf6b45f8784d23a0eb67263c040476"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "a81fe0cf5b8d269e1ea9156661847cbe"
server_key_iv: "d8fe1ce1b09299f6a03dd4c869e2beec"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.841823 6983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:22.851168 6983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:22.853015 6983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:22.853171 6983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:22.853394 6983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:22.853705 6983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:22.853739 6983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:22.853761 6983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:22.853808 6983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:22.860026 6983 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:45671
I20260807 20:08:22.860075 7122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:45671 every 8 connection(s)
I20260807 20:08:22.860404 6983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:22 dist-test-slave-1x4g krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:22.869225 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 6983
I20260807 20:08:22.869341 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:22.869546 3134 external_mini_cluster.cc:1428] Setting key 8235cae571a70cb434833f4c4bae5694
I20260807 20:08:22.870294 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:22.861117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:37079 (local address 127.3.15.190:35937)
0807 20:08:22.861178 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:22.861180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:22.861377 (+ 197us) server_negotiation.cc:407] Connection header received
0807 20:08:22.862382 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:22.862383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:22.862400 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:22.862461 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:22.863073 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:22.863709 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:22.864912 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:22.865137 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:22.865226 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:22.867236 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:22.867238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:22.867238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:22.867246 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:22.868544 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:22.869110 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:22.869112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:22.869112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:22.869185 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:22.869511 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:22.869512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:22.869513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:22.869680 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:22.869753 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:22.870076 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 20:08:22.870178 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:08:22.870595 7125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:22.861107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35937 (local address 127.3.15.129:37079)
0807 20:08:22.861257 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:22.861294 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:22.862261 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:22.862535 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:22.862539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:22.862783 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:22.862975 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:22.862979 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:22.863810 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:22.863811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:22.864816 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:22.864819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:22.865235 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:22.865237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:22.865364 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:22.865811 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:22.865819 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:22.867134 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:22.868658 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:22.868665 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:22.868667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:22.869005 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:22.869262 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:22.869263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:22.869264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:22.869444 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:22.869744 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:22.869751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:22.869928 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 20:08:22.870054 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 20:08:22.870228 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260807 20:08:22.871095 7123 heartbeater.cc:347] Connected to a master server at 127.3.15.190:35937
I20260807 20:08:22.871219 7123 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:22.871424 7123 heartbeater.cc:511] Master 127.3.15.190:35937 requested a full tablet report, sending...
I20260807 20:08:22.871997 6918 ts_manager.cc:194] Registered new tserver with Master: 78cf6b45f8784d23a0eb67263c040476 (127.3.15.129:45671)
I20260807 20:08:22.872653 6918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:37079
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:22.896103 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35937
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:36433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:22.978750 7129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:22.978943 7129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:22.978966 7129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:22.980098 7129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:22.980140 7129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:22.980166 7129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:22.981760 7129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:22.982014 7129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:22.982249 7129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:22.987423 7136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:22.987423 7134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:22.987458 7138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:22.987522 7129 server_base.cc:1034] running on GCE node
I20260807 20:08:22.987855 7129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:22.988129 7129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:22.989296 7129 hybrid_clock.cc:648] HybridClock initialized: now 1786133302989170 us; error 156 us; skew 500 ppm
Aug 07 20:08:22 dist-test-slave-1x4g krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:22.991844 7129 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:22.992301 7129 webserver.cc:460] Webserver started at http://127.3.15.130:34401/ using document root <none> and password file <none>
I20260807 20:08:22.992542 7129 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:22.992610 7129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:22.992766 7129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:22.993829 7129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "91eff7f508d342daaa6f0fa4405a0c95"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "7d55974c8b9008d88a6e1cf8cca905b1"
server_key_iv: "77137bdacdd493988a7499ff24e89a73"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.994202 7129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "91eff7f508d342daaa6f0fa4405a0c95"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "7d55974c8b9008d88a6e1cf8cca905b1"
server_key_iv: "77137bdacdd493988a7499ff24e89a73"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.995541 7129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:08:22.996304 7145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:22.996488 7129 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:22.996588 7129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "91eff7f508d342daaa6f0fa4405a0c95"
format_stamp: "Formatted at 2026-08-07 20:08:22 on dist-test-slave-1x4g"
server_key: "7d55974c8b9008d88a6e1cf8cca905b1"
server_key_iv: "77137bdacdd493988a7499ff24e89a73"
server_key_version: "encryptionkey@0"
I20260807 20:08:22.996680 7129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:23.009310 7129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:23.011045 7129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:23.011200 7129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:23.011417 7129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:23.011718 7129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:23.011775 7129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:23.011814 7129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:23.011842 7129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:23.017617 7129 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:42325
I20260807 20:08:23.017654 7268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:42325 every 8 connection(s)
I20260807 20:08:23.017971 7129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:23.021402 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 7129
I20260807 20:08:23.021540 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:23.021756 3134 external_mini_cluster.cc:1428] Setting key 577fbd66a1ba22f2a04436d2e6832f9b
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133302, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:23.029187 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:23.018753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:60387 (local address 127.3.15.190:35937)
0807 20:08:23.018832 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:23.018835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:23.019008 (+ 173us) server_negotiation.cc:407] Connection header received
0807 20:08:23.020008 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:23.020010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:23.020024 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:23.020075 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:23.020698 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.021369 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.022700 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.022993 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.023072 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:23.025907 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:23.025908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:23.025909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:23.025915 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:23.027472 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.028003 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.028004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.028004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.028056 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.028381 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.028383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.028383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.028522 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:23.028637 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:23.029032 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 20:08:23.029117 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:08:23.029445 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:23.018722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35937 (local address 127.3.15.130:60387)
0807 20:08:23.018894 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:23.018929 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:23.019882 (+ 953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:23.020150 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:23.020154 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:23.020404 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:23.020583 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:23.020588 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.021512 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.021514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:23.022621 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:23.022624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.023245 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.023247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:23.023369 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:23.023928 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:23.023938 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:23.025814 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:23.027566 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:23.027574 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:23.027576 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:23.027926 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:23.028124 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:23.028125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:23.028126 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:23.028313 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:23.028641 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:23.028649 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:23.028864 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 20:08:23.028992 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 20:08:23.029187 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260807 20:08:23.030161 7269 heartbeater.cc:347] Connected to a master server at 127.3.15.190:35937
I20260807 20:08:23.030287 7269 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:23.030488 7269 heartbeater.cc:511] Master 127.3.15.190:35937 requested a full tablet report, sending...
I20260807 20:08:23.030946 6918 ts_manager.cc:194] Registered new tserver with Master: 91eff7f508d342daaa6f0fa4405a0c95 (127.3.15.130:42325)
I20260807 20:08:23.031490 6918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:60387
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:23.050182 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35937
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:36433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:23.137212 7275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:23.137434 7275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:23.137458 7275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:23.138613 7275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:23.138655 7275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:23.138687 7275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:23.140230 7275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:23.140470 7275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:23.140726 7275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:23.146216 7280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:23.146226 7281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:23.146234 7284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:23.146518 7275 server_base.cc:1034] running on GCE node
I20260807 20:08:23.146728 7275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:23.146978 7275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:23.148149 7275 hybrid_clock.cc:648] HybridClock initialized: now 1786133303148130 us; error 30 us; skew 500 ppm
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133303, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:23.150934 7275 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:23.151372 7275 webserver.cc:460] Webserver started at http://127.3.15.131:38287/ using document root <none> and password file <none>
I20260807 20:08:23.151611 7275 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:23.151683 7275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:23.151844 7275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:23.153020 7275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "471fd2dd23bb44218e8b9ff5bae3d2b6"
format_stamp: "Formatted at 2026-08-07 20:08:23 on dist-test-slave-1x4g"
server_key: "79b6ded6a24f2df2334fa8a81c2dac61"
server_key_iv: "aefc93d7f365c1fe384b0b43cebb2c9a"
server_key_version: "encryptionkey@0"
I20260807 20:08:23.153395 7275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "471fd2dd23bb44218e8b9ff5bae3d2b6"
format_stamp: "Formatted at 2026-08-07 20:08:23 on dist-test-slave-1x4g"
server_key: "79b6ded6a24f2df2334fa8a81c2dac61"
server_key_iv: "aefc93d7f365c1fe384b0b43cebb2c9a"
server_key_version: "encryptionkey@0"
I20260807 20:08:23.154769 7275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:23.155459 7292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:23.155629 7275 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:23.155737 7275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "471fd2dd23bb44218e8b9ff5bae3d2b6"
format_stamp: "Formatted at 2026-08-07 20:08:23 on dist-test-slave-1x4g"
server_key: "79b6ded6a24f2df2334fa8a81c2dac61"
server_key_iv: "aefc93d7f365c1fe384b0b43cebb2c9a"
server_key_version: "encryptionkey@0"
I20260807 20:08:23.155835 7275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:23.170507 7275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:23.173121 7275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:23.173295 7275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:23.173623 7275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:23.173965 7275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:23.174024 7275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:23.174067 7275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:23.174096 7275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:23.180236 7275 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:45329
I20260807 20:08:23.180284 7413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:45329 every 8 connection(s)
I20260807 20:08:23.180612 7275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:23.186286 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 7275
I20260807 20:08:23.186360 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:23.186553 3134 external_mini_cluster.cc:1428] Setting key 539cf4fc886507d8196582823607864b
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133303, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:23.190096 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:23.181362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:44623 (local address 127.3.15.190:35937)
0807 20:08:23.181411 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:23.181454 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:23.181641 (+ 187us) server_negotiation.cc:407] Connection header received
0807 20:08:23.182653 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:23.182655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:23.182669 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:23.182722 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:23.183393 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.183938 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.185025 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.185226 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.185316 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:23.187671 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:23.187673 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:23.187673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:23.187680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:23.188942 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.189315 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.189316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.189316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.189369 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.189582 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.189583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.189583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.189684 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:23.189737 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:23.189969 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 20:08:23.190033 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:08:23.190263 7417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:23.181329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35937 (local address 127.3.15.131:44623)
0807 20:08:23.181492 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:23.181545 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:23.182532 (+ 987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:23.182808 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:23.182812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:23.183088 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:23.183290 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:23.183294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.184040 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.184041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:23.184935 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:23.184938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.185326 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.185327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:23.185466 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:23.185964 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:23.185973 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:23.187552 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:23.189012 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:23.189018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:23.189020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:23.189250 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:23.189427 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:23.189427 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:23.189428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:23.189527 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:23.189739 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:23.189743 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:23.189857 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 20:08:23.189939 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 20:08:23.190058 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260807 20:08:23.190811 7415 heartbeater.cc:347] Connected to a master server at 127.3.15.190:35937
I20260807 20:08:23.190908 7415 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:23.191083 7415 heartbeater.cc:511] Master 127.3.15.190:35937 requested a full tablet report, sending...
I20260807 20:08:23.191473 6918 ts_manager.cc:194] Registered new tserver with Master: 471fd2dd23bb44218e8b9ff5bae3d2b6 (127.3.15.131:45329)
I20260807 20:08:23.191808 6918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:44623
I20260807 20:08:23.198256 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[6870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:23.222334 7422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:23.214673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.3.15.129:45671)
0807 20:08:23.214853 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:23.214858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:23.214869 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:23.214897 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:23.214899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:23.214913 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:23.214981 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:23.215528 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.216265 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.217079 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.217411 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.217478 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:23.219686 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:23.219693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:23.219693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:23.219701 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:23.221202 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.221513 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.221516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.221516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.221585 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.221800 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.221800 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.221801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.221938 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:23.222045 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:23.222131 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 20:08:23.222196 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":91,"threads_started":1}
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[6870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:23.230379 6961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:23.223737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.3.15.190:35937)
0807 20:08:23.223805 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:23.223807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:23.223815 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:23.223866 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:23.223868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:23.223877 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:23.223919 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:23.224342 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.224899 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.225519 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:23.225707 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:23.225757 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:23.228023 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:23.228024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:23.228024 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:23.228038 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:23.229395 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.229702 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.229702 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.229703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.229753 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:23.229913 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:23.229914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:23.229914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:23.230056 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:23.230147 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:23.230215 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 20:08:23.230278 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 20:08:23.230613 6918 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40396
I20260807 20:08:23.230947 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6983
I20260807 20:08:23.236702 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 7129
I20260807 20:08:23.243583 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 7275
I20260807 20:08:23.249845 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 6884
2026-08-07T20:08:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1915 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[7429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[7429](info): set up 2 sockets
Aug 07 20:08:23 dist-test-slave-1x4g krb5kdc[7429](info): commencing operation
krb5kdc: starting...
W20260807 20:08:24.318758 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:24.444859 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:38541
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:38541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:24.530782 7445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:24.531008 7445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:24.531030 7445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:24.532155 7445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:24.532192 7445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:24.532207 7445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:24.532219 7445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:24.532231 7445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:24.533771 7445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:38541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:38541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:24.533983 7445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:24.534157 7445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:24.539212 7453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:24.539197 7451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:24.539523 7445 server_base.cc:1034] running on GCE node
W20260807 20:08:24.539256 7450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:24.539865 7445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:24.540123 7445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:24.541260 7445 hybrid_clock.cc:648] HybridClock initialized: now 1786133304541242 us; error 35 us; skew 500 ppm
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:24.543776 7445 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:24.544199 7445 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:24.544272 7445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:24.544414 7445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:24.545583 7445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "6c8b2e574bae4654a17e8d842835ea8b"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "54570264cb4f05219e432ec98b882cda"
server_key_iv: "1b6b8cd50d9d3d6897467ab840c7e717"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.545948 7445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "6c8b2e574bae4654a17e8d842835ea8b"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "54570264cb4f05219e432ec98b882cda"
server_key_iv: "1b6b8cd50d9d3d6897467ab840c7e717"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.547326 7445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:08:24.548108 7459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.548372 7445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:24.548477 7445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "6c8b2e574bae4654a17e8d842835ea8b"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "54570264cb4f05219e432ec98b882cda"
server_key_iv: "1b6b8cd50d9d3d6897467ab840c7e717"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.548555 7445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:24.562139 7445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:24.565364 7445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:24.565543 7445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:24.571578 7445 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:38541
I20260807 20:08:24.571637 7516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:38541 every 8 connection(s)
I20260807 20:08:24.571950 7445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:24.573369 7518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:24.575904 7518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b: Bootstrap starting.
I20260807 20:08:24.576514 7518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:24.576906 7518 log.cc:824] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:24.577587 7518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b: No bootstrap required, opened a new log
I20260807 20:08:24.579171 7518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8b2e574bae4654a17e8d842835ea8b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38541 } }
I20260807 20:08:24.579331 7518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:24.579380 7518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8b2e574bae4654a17e8d842835ea8b, State: Initialized, Role: FOLLOWER
I20260807 20:08:24.579524 7518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8b2e574bae4654a17e8d842835ea8b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38541 } }
I20260807 20:08:24.579622 7518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:24.579663 7518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:24.579712 7518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:24.579793 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 7445
I20260807 20:08:24.579916 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:24.580134 3134 external_mini_cluster.cc:1428] Setting key 7e7d284ee1652f0bb46904e3a1a206f0
I20260807 20:08:24.580890 7518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8b2e574bae4654a17e8d842835ea8b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38541 } }
I20260807 20:08:24.581007 7518 leader_election.cc:302] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c8b2e574bae4654a17e8d842835ea8b; no voters:
I20260807 20:08:24.581182 7518 leader_election.cc:288] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:24.581252 7523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:24.581398 7523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c8b2e574bae4654a17e8d842835ea8b, State: Running, Role: LEADER
I20260807 20:08:24.581452 7518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:24.581503 7523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8b2e574bae4654a17e8d842835ea8b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38541 } }
I20260807 20:08:24.581806 7523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c8b2e574bae4654a17e8d842835ea8b. Latest consensus state: current_term: 1 leader_uuid: "6c8b2e574bae4654a17e8d842835ea8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8b2e574bae4654a17e8d842835ea8b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38541 } } }
I20260807 20:08:24.581816 7525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c8b2e574bae4654a17e8d842835ea8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c8b2e574bae4654a17e8d842835ea8b" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 38541 } } }
I20260807 20:08:24.581898 7523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:24.581904 7525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:24.582157 7530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:24.582713 7530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:24.584319 7530 catalog_manager.cc:1347] Generated new cluster ID: 9182ef81a6204a8195d9d2084a63765b
I20260807 20:08:24.584414 7530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:24.591715 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:24.580666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.3.15.190:38541)
0807 20:08:24.580895 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:24.580906 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:24.580943 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:08:24.581065 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:24.581073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:24.581136 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:24.581363 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:24.582376 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.583599 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.584647 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.585314 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.585386 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:24.587819 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:24.587824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:24.587824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:24.587837 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:24.589936 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:24.590372 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:24.590375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:24.590376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:24.590509 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:24.590747 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:24.590748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:24.590748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:24.591077 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:24.591238 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:24.591347 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 20:08:24.591498 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":114,"threads_started":1}
I20260807 20:08:24.592823 7530 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:24.593326 7530 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:24.597888 7530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b: Generated new TSK 0
I20260807 20:08:24.598064 7530 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:24.631904 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38541
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:24.715925 7545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:24.716140 7545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:24.716166 7545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:24.717368 7545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:24.717413 7545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:24.717440 7545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:24.718891 7545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:24.719084 7545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:24.719264 7545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:08:24.724309 7545 server_base.cc:1034] running on GCE node
W20260807 20:08:24.724345 7553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:24.724363 7551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:24.724421 7550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:24.724663 7545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:24.724936 7545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:24.726083 7545 hybrid_clock.cc:648] HybridClock initialized: now 1786133304726072 us; error 30 us; skew 500 ppm
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:24.728634 7545 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:24.729060 7545 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:24.729139 7545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:24.729286 7545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:24.730348 7545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "9a0b2e451bbe4316be1e3eb53d489047"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "31193cd8a23543d25a617578432ebe7f"
server_key_iv: "ae928372f1092c277bdd8d6671eb1a72"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.730710 7545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "9a0b2e451bbe4316be1e3eb53d489047"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "31193cd8a23543d25a617578432ebe7f"
server_key_iv: "ae928372f1092c277bdd8d6671eb1a72"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.732059 7545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:24.732880 7560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.733048 7545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:24.733162 7545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "9a0b2e451bbe4316be1e3eb53d489047"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "31193cd8a23543d25a617578432ebe7f"
server_key_iv: "ae928372f1092c277bdd8d6671eb1a72"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.733242 7545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:24.756740 7545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:24.758785 7545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:24.758950 7545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:24.759210 7545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:24.759541 7545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:24.759593 7545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.759632 7545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:24.759672 7545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.765522 7545 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:42167
I20260807 20:08:24.765568 7688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:42167 every 8 connection(s)
I20260807 20:08:24.765847 7545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:24.767282 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 7545
I20260807 20:08:24.767391 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:24.767637 3134 external_mini_cluster.cc:1428] Setting key 1b3316f2881f69f8704b5f5269049455
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:24.776741 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:24.766566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:58905 (local address 127.3.15.190:38541)
0807 20:08:24.766634 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:24.766636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:24.766852 (+ 216us) server_negotiation.cc:407] Connection header received
0807 20:08:24.767532 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:24.767533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:24.767548 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:24.767590 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:24.768931 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.769431 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.770699 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.770910 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.770988 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:24.773574 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:24.773575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:24.773575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:24.773584 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:24.775031 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:24.775556 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:24.775557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:24.775557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:24.775624 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:24.775968 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:24.775970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:24.775970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:24.776133 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:24.776183 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:24.776586 (+ 403us) server_negotiation.cc:299] Negotiation successful
0807 20:08:24.776672 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:24.777048 7691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:24.766545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38541 (local address 127.3.15.129:58905)
0807 20:08:24.766729 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:24.766771 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:24.767408 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:24.767674 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:24.767678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:24.768072 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:24.768830 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:24.768838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.769520 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.769521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:24.770609 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:24.770612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.770996 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.770998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:24.771133 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:24.771652 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:24.771660 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:24.773480 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:24.775117 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:24.775124 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:24.775127 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:24.775482 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:24.775689 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:24.775690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:24.775691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:24.775866 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:24.776237 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:24.776244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:24.776414 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 20:08:24.776568 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 20:08:24.776761 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260807 20:08:24.777618 7689 heartbeater.cc:347] Connected to a master server at 127.3.15.190:38541
I20260807 20:08:24.777735 7689 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
I20260807 20:08:24.777891 7689 heartbeater.cc:511] Master 127.3.15.190:38541 requested a full tablet report, sending...
I20260807 20:08:24.778410 7480 ts_manager.cc:194] Registered new tserver with Master: 9a0b2e451bbe4316be1e3eb53d489047 (127.3.15.129:42167)
I20260807 20:08:24.779116 7480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:58905
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:24.795707 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38541
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:24.881942 7694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:24.882201 7694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:24.882242 7694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:24.883561 7694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:24.883630 7694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:24.883678 7694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:24.885316 7694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:24.885610 7694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:24.885857 7694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:24.891125 7702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:24.891197 7694 server_base.cc:1034] running on GCE node
W20260807 20:08:24.891129 7699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:24.891105 7700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:24.891539 7694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:24.891793 7694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:24.892933 7694 hybrid_clock.cc:648] HybridClock initialized: now 1786133304892914 us; error 35 us; skew 500 ppm
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:24.895612 7694 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:24.896023 7694 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:24.896096 7694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:24.896245 7694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:24.897437 7694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "5df2519808144c74bdb387b5a8c2910b"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "72c198849ec57eae79256111bfb8fb0a"
server_key_iv: "c3031ee7b0d6a25dfe2ec8eff060aa81"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.897814 7694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "5df2519808144c74bdb387b5a8c2910b"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "72c198849ec57eae79256111bfb8fb0a"
server_key_iv: "c3031ee7b0d6a25dfe2ec8eff060aa81"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.899147 7694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:24.899884 7709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.900058 7694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:24.900163 7694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "5df2519808144c74bdb387b5a8c2910b"
format_stamp: "Formatted at 2026-08-07 20:08:24 on dist-test-slave-1x4g"
server_key: "72c198849ec57eae79256111bfb8fb0a"
server_key_iv: "c3031ee7b0d6a25dfe2ec8eff060aa81"
server_key_version: "encryptionkey@0"
I20260807 20:08:24.900236 7694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:24.908281 7694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:24.910356 7694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:24.910531 7694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:24.910775 7694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:24.911178 7694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:24.911232 7694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.911274 7694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:24.911305 7694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:24.917536 7694 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:42581
I20260807 20:08:24.917572 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:42581 every 8 connection(s)
I20260807 20:08:24.917876 7694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:24.921259 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 7694
I20260807 20:08:24.921392 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:24.921582 3134 external_mini_cluster.cc:1428] Setting key 58ebb2aeb4ef5484530f4b3b9592d120
Aug 07 20:08:24 dist-test-slave-1x4g krb5kdc[7429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133304, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:24.928987 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:24.918610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:39625 (local address 127.3.15.190:38541)
0807 20:08:24.918676 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:24.918679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:24.918936 (+ 257us) server_negotiation.cc:407] Connection header received
0807 20:08:24.919430 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:24.919431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:24.919442 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:24.919483 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:24.920457 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.920984 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.922257 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.922506 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.922563 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:24.925485 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:24.925486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:24.925487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:24.925496 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:24.927139 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:24.927667 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:24.927668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:24.927668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:24.927732 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:24.928076 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:24.928077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:24.928077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:24.928203 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:24.928260 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:24.928659 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 20:08:24.928744 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 20:08:24.929178 7840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:24.918596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38541 (local address 127.3.15.130:39625)
0807 20:08:24.918800 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:24.918841 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:24.919333 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:24.919544 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:24.919547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:24.919816 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:24.920376 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:24.920382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.921084 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.921085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:24.922140 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:24.922143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:24.922594 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:24.922596 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:24.922758 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:24.923299 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:24.923311 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:24.925345 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:24.927236 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:24.927245 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:24.927248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:24.927582 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:24.927804 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:24.927806 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:24.927806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:24.927975 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:24.928275 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:24.928284 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:24.928483 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 20:08:24.928616 (+ 133us) client_negotiation.cc:240] Negotiation successful
0807 20:08:24.928860 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260807 20:08:24.929720 7837 heartbeater.cc:347] Connected to a master server at 127.3.15.190:38541
I20260807 20:08:24.929842 7837 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:24.930068 7837 heartbeater.cc:511] Master 127.3.15.190:38541 requested a full tablet report, sending...
I20260807 20:08:24.930526 7480 ts_manager.cc:194] Registered new tserver with Master: 5df2519808144c74bdb387b5a8c2910b (127.3.15.130:42581)
I20260807 20:08:24.930912 7480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:39625
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:24.951498 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38541
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:25.037155 7843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:25.037353 7843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:25.037376 7843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:25.038529 7843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:25.038571 7843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:25.038631 7843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:25.040138 7843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:38541
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:25.040378 7843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:25.040606 7843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:25.045668 7849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.045688 7848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.045688 7851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.045782 7843 server_base.cc:1034] running on GCE node
I20260807 20:08:25.046154 7843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:25.046397 7843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:25.047559 7843 hybrid_clock.cc:648] HybridClock initialized: now 1786133305047547 us; error 33 us; skew 500 ppm
Aug 07 20:08:25 dist-test-slave-1x4g krb5kdc[7429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:25.050413 7843 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:25.050834 7843 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:25.050885 7843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:25.051034 7843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:25.052342 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "2e119505e1f84a998479c7839a79f923"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "3a6e2b6e6212e3f02e1af4a575500b53"
server_key_iv: "62f139c8238de422dda79d9a4ccb2966"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.052731 7843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "2e119505e1f84a998479c7839a79f923"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "3a6e2b6e6212e3f02e1af4a575500b53"
server_key_iv: "62f139c8238de422dda79d9a4ccb2966"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.054150 7843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:25.054854 7857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.055022 7843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:25.055119 7843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "2e119505e1f84a998479c7839a79f923"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "3a6e2b6e6212e3f02e1af4a575500b53"
server_key_iv: "62f139c8238de422dda79d9a4ccb2966"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.055189 7843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:25.066275 7843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:25.068620 7843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:25.068739 7843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:25.068987 7843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:25.069303 7843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:25.069337 7843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.069376 7843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:25.069402 7843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.075112 7843 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:46229
I20260807 20:08:25.075181 7978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:46229 every 8 connection(s)
I20260807 20:08:25.075453 7843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:25.077284 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 7843
I20260807 20:08:25.077400 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:25.077610 3134 external_mini_cluster.cc:1428] Setting key 104401444838c9da0430de8f5f7a2179
Aug 07 20:08:25 dist-test-slave-1x4g krb5kdc[7429](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133305, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:25.085613 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.076190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:34955 (local address 127.3.15.190:38541)
0807 20:08:25.076252 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.076254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.076422 (+ 168us) server_negotiation.cc:407] Connection header received
0807 20:08:25.077048 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.077050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.077062 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.077129 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.078473 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.079071 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.080025 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.080208 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.080259 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.082580 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.082582 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.082582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:25.082589 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.084164 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:25.084533 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:25.084534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:25.084534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:25.084600 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:25.084996 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:25.084997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:25.084997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:25.085109 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.085165 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.085468 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.085530 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:08:25.085786 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.076149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:38541 (local address 127.3.15.131:34955)
0807 20:08:25.076301 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.076342 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.076915 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.077230 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.077235 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.077622 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.078355 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.078363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.079150 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.079151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.079952 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.079956 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.080360 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.080361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.080487 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.081048 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:25.081057 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.082499 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.084240 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:25.084247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:25.084248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:25.084469 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:25.084657 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:25.084658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:25.084658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:25.084759 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:25.085198 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.085204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.085330 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.085434 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.085573 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260807 20:08:25.086365 7979 heartbeater.cc:347] Connected to a master server at 127.3.15.190:38541
I20260807 20:08:25.086465 7979 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:25.086585 7979 heartbeater.cc:511] Master 127.3.15.190:38541 requested a full tablet report, sending...
I20260807 20:08:25.087024 7480 ts_manager.cc:194] Registered new tserver with Master: 2e119505e1f84a998479c7839a79f923 (127.3.15.131:46229)
I20260807 20:08:25.087473 7480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:34955
I20260807 20:08:25.089478 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:08:25.096509 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.090826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58390 (local address 127.3.15.190:38541)
0807 20:08:25.090870 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.090874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.091004 (+ 130us) server_negotiation.cc:407] Connection header received
0807 20:08:25.091045 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.091046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.091054 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.091107 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.091680 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.092141 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.092931 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.093127 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.093174 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.094242 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.094243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.094243 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:25.094249 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.095488 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:25.095816 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:25.095816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:25.095817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:25.095866 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:25.096068 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:25.096068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:25.096068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:25.096215 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.096290 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.096385 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.096445 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:25.098309 7480 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:25.099906 7480 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:25.106715 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.102735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:42167 (local address 127.0.0.1:35152)
0807 20:08:25.103188 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.103197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.103253 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.103487 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.103488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.103489 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:25.103517 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.103518 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.104478 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.104479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.105567 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.105570 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.106379 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.106380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.106466 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.106480 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.106561 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.106616 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":100,"threads_started":1}
I20260807 20:08:25.106719 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.102898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.3.15.129:42167)
0807 20:08:25.103132 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.103137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.103267 (+ 130us) server_negotiation.cc:407] Connection header received
0807 20:08:25.103318 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.103320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.103336 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.103412 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:25.103586 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.104377 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.105664 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.106291 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.106368 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.106429 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.106586 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.106632 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":88,"threads_started":1}
I20260807 20:08:25.107113 7993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.102616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.3.15.131:46229)
0807 20:08:25.102908 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.102914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.102925 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:25.102968 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.102970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.102988 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.103064 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:25.103401 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.104576 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.106029 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.106643 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.106705 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.106761 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.106980 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.107022 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":99,"threads_started":1}
I20260807 20:08:25.107062 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.102506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46229 (local address 127.0.0.1:41522)
0807 20:08:25.102650 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.102659 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.102694 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.103191 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.103199 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.103201 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:25.103305 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.103308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.104676 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.104677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.105939 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.105941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.106796 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.106798 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.106879 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.106884 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.106953 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.106998 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":56,"thread_start_us":96,"threads_started":1}
I20260807 20:08:25.107668 7990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.102252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42581 (local address 127.0.0.1:46960)
0807 20:08:25.102387 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.102431 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.102507 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.103217 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.103219 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.103221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:25.103309 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.103311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.104712 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.104714 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.105909 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.105913 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.107439 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.107440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.107504 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.107509 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.107561 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.107605 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":28,"thread_start_us":63,"threads_started":1}
I20260807 20:08:25.107699 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.102397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46960 (local address 127.3.15.130:42581)
0807 20:08:25.102982 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.102988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.103000 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:08:25.103045 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.103047 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.103064 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.103394 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:25.103490 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.104591 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.106005 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.106756 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.107439 (+ 683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.107496 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.107590 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.107638 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":105,"threads_started":1}
I20260807 20:08:25.107582 7618 tablet_service.cc:1467] Processing CreateTablet for tablet 5572f8070b56413083422f8a75cd6b76 (DEFAULT_TABLE table=test-table [id=f651e42d91d94263acdaae94503d25c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:25.108007 7618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5572f8070b56413083422f8a75cd6b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.108402 7764 tablet_service.cc:1467] Processing CreateTablet for tablet 5572f8070b56413083422f8a75cd6b76 (DEFAULT_TABLE table=test-table [id=f651e42d91d94263acdaae94503d25c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:25.108736 7764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5572f8070b56413083422f8a75cd6b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.109331 7908 tablet_service.cc:1467] Processing CreateTablet for tablet 5572f8070b56413083422f8a75cd6b76 (DEFAULT_TABLE table=test-table [id=f651e42d91d94263acdaae94503d25c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:25.109647 7908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5572f8070b56413083422f8a75cd6b76. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.111635 7996 tablet_bootstrap.cc:492] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: Bootstrap starting.
I20260807 20:08:25.112164 7997 tablet_bootstrap.cc:492] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Bootstrap starting.
I20260807 20:08:25.112416 7996 tablet_bootstrap.cc:654] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.112429 7999 tablet_bootstrap.cc:492] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Bootstrap starting.
I20260807 20:08:25.112871 7997 tablet_bootstrap.cc:654] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.113158 7996 log.cc:824] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.113276 7999 tablet_bootstrap.cc:654] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.113494 7997 log.cc:824] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.113864 7999 log.cc:824] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.114293 7997 tablet_bootstrap.cc:492] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: No bootstrap required, opened a new log
I20260807 20:08:25.114372 7997 ts_tablet_manager.cc:1397] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:08:25.114578 7999 tablet_bootstrap.cc:492] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: No bootstrap required, opened a new log
I20260807 20:08:25.114619 7996 tablet_bootstrap.cc:492] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: No bootstrap required, opened a new log
I20260807 20:08:25.114660 7999 ts_tablet_manager.cc:1397] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:08:25.114689 7996 ts_tablet_manager.cc:1397] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:08:25.115976 7999 raft_consensus.cc:348] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.116138 7999 raft_consensus.cc:374] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.116184 7999 raft_consensus.cc:729] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e119505e1f84a998479c7839a79f923, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.116302 7999 consensus_queue.cc:260] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.116420 7997 raft_consensus.cc:348] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.116580 7999 ts_tablet_manager.cc:1428] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:08:25.116600 7997 raft_consensus.cc:374] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.116663 7997 raft_consensus.cc:729] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5df2519808144c74bdb387b5a8c2910b, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.116719 7979 heartbeater.cc:503] Master 127.3.15.190:38541 was elected leader, sending a full tablet report...
I20260807 20:08:25.116636 7996 raft_consensus.cc:348] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.116819 7996 raft_consensus.cc:374] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.116802 7997 consensus_queue.cc:260] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.116858 7996 raft_consensus.cc:729] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0b2e451bbe4316be1e3eb53d489047, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.116994 7996 consensus_queue.cc:260] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.117069 7997 ts_tablet_manager.cc:1428] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 20:08:25.117182 7837 heartbeater.cc:503] Master 127.3.15.190:38541 was elected leader, sending a full tablet report...
I20260807 20:08:25.117277 7996 ts_tablet_manager.cc:1428] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 20:08:25.117435 7689 heartbeater.cc:503] Master 127.3.15.190:38541 was elected leader, sending a full tablet report...
W20260807 20:08:25.168440 7838 tablet.cc:2369] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:25.252918 8003 raft_consensus.cc:482] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:25.253095 8003 raft_consensus.cc:504] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.253557 8003 leader_election.cc:288] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5df2519808144c74bdb387b5a8c2910b (127.3.15.130:42581), 9a0b2e451bbe4316be1e3eb53d489047 (127.3.15.129:42167)
I20260807 20:08:25.256814 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.253852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:49921 (local address 127.3.15.130:42581)
0807 20:08:25.253912 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.253915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.253926 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:25.253997 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.253998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.254012 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.254081 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:25.254317 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.254877 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.256022 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.256401 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.256487 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.256498 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.256672 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.256715 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:08:25.256826 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.253750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42581 (local address 127.3.15.131:49921)
0807 20:08:25.253810 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.253826 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.253881 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.254137 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.254138 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.254140 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:25.254213 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.254230 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.254962 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.254964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.255901 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.255904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.256487 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.256488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.256580 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.256590 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.256649 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.256713 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":26}
I20260807 20:08:25.256846 7995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.253987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:50451 (local address 127.3.15.129:42167)
0807 20:08:25.254078 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.254081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.254243 (+ 162us) server_negotiation.cc:407] Connection header received
0807 20:08:25.254280 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.254281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.254295 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.254360 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:25.254498 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.255031 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.255986 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.256399 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.256507 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.256517 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.256697 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.256743 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:08:25.256837 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.253983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:42167 (local address 127.3.15.131:50451)
0807 20:08:25.254174 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.254184 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.254235 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.254368 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.254369 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.254371 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:25.254435 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.254436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.255122 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.255123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.255896 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:25.255898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.256513 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.256514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:25.256597 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.256600 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.256680 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.256719 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":5,"thread_start_us":117,"threads_started":1}
I20260807 20:08:25.257244 7787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5572f8070b56413083422f8a75cd6b76" candidate_uuid: "2e119505e1f84a998479c7839a79f923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df2519808144c74bdb387b5a8c2910b" is_pre_election: true
I20260807 20:08:25.257431 7787 raft_consensus.cc:2454] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e119505e1f84a998479c7839a79f923 in term 0.
I20260807 20:08:25.257719 7859 leader_election.cc:302] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e119505e1f84a998479c7839a79f923, 5df2519808144c74bdb387b5a8c2910b; no voters:
I20260807 20:08:25.257817 7640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5572f8070b56413083422f8a75cd6b76" candidate_uuid: "2e119505e1f84a998479c7839a79f923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0b2e451bbe4316be1e3eb53d489047" is_pre_election: true
I20260807 20:08:25.257890 8003 raft_consensus.cc:2781] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:25.257952 8003 raft_consensus.cc:482] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:25.257994 8003 raft_consensus.cc:3037] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:25.258019 7640 raft_consensus.cc:2454] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e119505e1f84a998479c7839a79f923 in term 0.
I20260807 20:08:25.258816 8003 raft_consensus.cc:504] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.258991 8003 leader_election.cc:288] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [CANDIDATE]: Term 1 election: Requested vote from peers 5df2519808144c74bdb387b5a8c2910b (127.3.15.130:42581), 9a0b2e451bbe4316be1e3eb53d489047 (127.3.15.129:42167)
I20260807 20:08:25.259182 7787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5572f8070b56413083422f8a75cd6b76" candidate_uuid: "2e119505e1f84a998479c7839a79f923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5df2519808144c74bdb387b5a8c2910b"
I20260807 20:08:25.259212 7640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5572f8070b56413083422f8a75cd6b76" candidate_uuid: "2e119505e1f84a998479c7839a79f923" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0b2e451bbe4316be1e3eb53d489047"
I20260807 20:08:25.259274 7640 raft_consensus.cc:3037] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:25.259268 7787 raft_consensus.cc:3037] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:25.259996 7640 raft_consensus.cc:2454] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e119505e1f84a998479c7839a79f923 in term 1.
I20260807 20:08:25.260069 7787 raft_consensus.cc:2454] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e119505e1f84a998479c7839a79f923 in term 1.
I20260807 20:08:25.260210 7859 leader_election.cc:302] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e119505e1f84a998479c7839a79f923, 9a0b2e451bbe4316be1e3eb53d489047; no voters:
I20260807 20:08:25.260326 8003 raft_consensus.cc:2781] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:25.260535 8003 raft_consensus.cc:686] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 1 LEADER]: Becoming Leader. State: Replica: 2e119505e1f84a998479c7839a79f923, State: Running, Role: LEADER
I20260807 20:08:25.260650 8003 consensus_queue.cc:237] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } }
I20260807 20:08:25.261582 7480 catalog_manager.cc:5495] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e119505e1f84a998479c7839a79f923 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "2e119505e1f84a998479c7839a79f923" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e119505e1f84a998479c7839a79f923" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 } health_report { overall_health: UNKNOWN } } }
W20260807 20:08:25.266494 7690 tablet.cc:2369] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:25.286530 7993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.284140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41524 (local address 127.3.15.131:46229)
0807 20:08:25.284194 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.284197 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.284206 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:25.284248 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.284249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.284277 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.284355 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:25.284476 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.284992 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.285742 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:25.285933 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:25.286006 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:25.286179 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:25.286283 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:25.286374 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.286394 (+ 20us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.286414 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:08:25.288439 7787 raft_consensus.cc:1261] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 1 FOLLOWER]: Refusing update from remote peer 2e119505e1f84a998479c7839a79f923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:25.288439 7640 raft_consensus.cc:1261] T 5572f8070b56413083422f8a75cd6b76 P 9a0b2e451bbe4316be1e3eb53d489047 [term 1 FOLLOWER]: Refusing update from remote peer 2e119505e1f84a998479c7839a79f923: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:25.288838 8009 consensus_queue.cc:1035] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0b2e451bbe4316be1e3eb53d489047" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:25.288985 8003 consensus_queue.cc:1035] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5df2519808144c74bdb387b5a8c2910b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:25.292995 8012 mvcc.cc:205] Tried to move back new op lower bound from 7316002018458525696 to 7316002018348478464. Current Snapshot: MvccSnapshot[applied={T|T < 7316002018458525696 or (T in {7316002018458525696})}]
I20260807 20:08:25.294075 7479 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:25.294164 7479 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:25.294938 7479 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c8b2e574bae4654a17e8d842835ea8b: Sending DeleteTablet for 3 replicas of tablet 5572f8070b56413083422f8a75cd6b76
I20260807 20:08:25.295286 7764 tablet_service.cc:1514] Processing DeleteTablet for tablet 5572f8070b56413083422f8a75cd6b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:25 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:46960
I20260807 20:08:25.295337 7618 tablet_service.cc:1514] Processing DeleteTablet for tablet 5572f8070b56413083422f8a75cd6b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:25 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:35152
I20260807 20:08:25.295543 7908 tablet_service.cc:1514] Processing DeleteTablet for tablet 5572f8070b56413083422f8a75cd6b76 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:25 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41522
I20260807 20:08:25.295784 8024 tablet_replica.cc:330] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: stopping tablet replica
I20260807 20:08:25.295842 8022 tablet_replica.cc:330] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: stopping tablet replica
I20260807 20:08:25.295931 8022 raft_consensus.cc:2229] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:25.295931 8024 raft_consensus.cc:2229] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:25.296036 8022 raft_consensus.cc:2258] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:25.296044 8024 raft_consensus.cc:2258] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:25.296118 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 7545
I20260807 20:08:25.296573 8024 ts_tablet_manager.cc:1905] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:25.298237 8024 ts_tablet_manager.cc:1918] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:25.298326 8024 log.cc:1196] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/5572f8070b56413083422f8a75cd6b76
I20260807 20:08:25.298565 8024 ts_tablet_manager.cc:1939] T 5572f8070b56413083422f8a75cd6b76 P 2e119505e1f84a998479c7839a79f923: Deleting consensus metadata
I20260807 20:08:25.298960 7462 catalog_manager.cc:4841] TS 2e119505e1f84a998479c7839a79f923 (127.3.15.131:46229): tablet 5572f8070b56413083422f8a75cd6b76 (table test-table [id=f651e42d91d94263acdaae94503d25c6]) successfully deleted
I20260807 20:08:25.300680 8022 ts_tablet_manager.cc:1905] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:25.302397 8022 ts_tablet_manager.cc:1918] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:25.302487 8022 log.cc:1196] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/5572f8070b56413083422f8a75cd6b76
I20260807 20:08:25.302754 8022 ts_tablet_manager.cc:1939] T 5572f8070b56413083422f8a75cd6b76 P 5df2519808144c74bdb387b5a8c2910b: Deleting consensus metadata
I20260807 20:08:25.303165 7462 catalog_manager.cc:4841] TS 5df2519808144c74bdb387b5a8c2910b (127.3.15.130:42581): tablet 5572f8070b56413083422f8a75cd6b76 (table test-table [id=f651e42d91d94263acdaae94503d25c6]) successfully deleted
W20260807 20:08:25.312448 7462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:42167 (error 108)
I20260807 20:08:25.312852 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 7694
I20260807 20:08:25.313086 7994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.312934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:42167 (local address 127.0.0.1:35158)
0807 20:08:25.312997 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.313045 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:42167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 20:08:25.313231 7462 catalog_manager.cc:4584] TS 9a0b2e451bbe4316be1e3eb53d489047 (127.3.15.129:42167): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5572f8070b56413083422f8a75cd6b76: Network error: Client connection negotiation failed: client connection to 127.3.15.129:42167: connect: Connection refused (error 111)
I20260807 20:08:25.321877 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 7843
I20260807 20:08:25.330086 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 7445
2026-08-07T20:08:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2079 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T20:08:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:25Z Disabled control of system clock
I20260807 20:08:25.359966 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:36531
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:40437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:36531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 20:08:25.444660 8029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:25.444917 8029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:25.444941 8029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:25.446111 8029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:25.446156 8029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:25.446168 8029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:25.446182 8029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:25.446193 8029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:25.447573 8029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:36531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:36531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:25.447824 8029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:25.448143 8029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:25.453053 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.453527 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.453527 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.453584 8029 server_base.cc:1034] running on GCE node
I20260807 20:08:25.453965 8029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:25.454268 8029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:25.455430 8029 hybrid_clock.cc:648] HybridClock initialized: now 1786133305455414 us; error 35 us; skew 500 ppm
I20260807 20:08:25.456728 8029 webserver.cc:460] Webserver started at http://127.3.15.190:34703/ using document root <none> and password file <none>
I20260807 20:08:25.457054 8029 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:25.457131 8029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:25.457275 8029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:25.458392 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "5a0f740d054e461181a26c7e41856446"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "95d7effa92491cd16075adf1c77f8620"
server_key_iv: "96ebe06fd5437d13aae59883fab3dc27"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.458779 8029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "5a0f740d054e461181a26c7e41856446"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "95d7effa92491cd16075adf1c77f8620"
server_key_iv: "96ebe06fd5437d13aae59883fab3dc27"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.460196 8029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:25.461090 8044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.461274 8029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:25.461385 8029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "5a0f740d054e461181a26c7e41856446"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "95d7effa92491cd16075adf1c77f8620"
server_key_iv: "96ebe06fd5437d13aae59883fab3dc27"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.461475 8029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:25.464908 8029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:25.465186 8029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:25.465323 8029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:25.469484 8029 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:36531
I20260807 20:08:25.469551 8102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:36531 every 8 connection(s)
I20260807 20:08:25.469873 8029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:25.470525 8103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.473443 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446: Bootstrap starting.
I20260807 20:08:25.473985 8103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.474238 8103 log.cc:824] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.474574 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8029
I20260807 20:08:25.474680 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:25.474879 3134 external_mini_cluster.cc:1428] Setting key bffdc5d0b86336fb4a5f87dbed55ac0a
I20260807 20:08:25.474989 8103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446: No bootstrap required, opened a new log
I20260807 20:08:25.477362 8103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f740d054e461181a26c7e41856446" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36531 } }
I20260807 20:08:25.477547 8103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.477701 8103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0f740d054e461181a26c7e41856446, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.477772 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.475437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.3.15.190:36531)
0807 20:08:25.475634 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.475641 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.475666 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:08:25.476268 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.476276 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.476317 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.476459 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.477226 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.477229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.477229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.477234 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.477277 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.477287 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.477357 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.477448 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.477524 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":77,"threads_started":1}
I20260807 20:08:25.477840 8103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f740d054e461181a26c7e41856446" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36531 } }
I20260807 20:08:25.477945 8103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:25.477995 8103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:25.478273 8103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:25.479192 8103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f740d054e461181a26c7e41856446" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36531 } }
I20260807 20:08:25.479305 8103 leader_election.cc:302] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0f740d054e461181a26c7e41856446; no voters:
I20260807 20:08:25.479461 8103 leader_election.cc:288] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:25.479523 8111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:25.479734 8111 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [term 1 LEADER]: Becoming Leader. State: Replica: 5a0f740d054e461181a26c7e41856446, State: Running, Role: LEADER
I20260807 20:08:25.479740 8103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:25.479843 8111 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f740d054e461181a26c7e41856446" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36531 } }
I20260807 20:08:25.480244 8112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0f740d054e461181a26c7e41856446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f740d054e461181a26c7e41856446" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36531 } } }
I20260807 20:08:25.480360 8112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:25.480599 8120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:25.480561 8113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0f740d054e461181a26c7e41856446. Latest consensus state: current_term: 1 leader_uuid: "5a0f740d054e461181a26c7e41856446" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0f740d054e461181a26c7e41856446" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 36531 } } }
I20260807 20:08:25.480886 8113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:25.481462 8120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:25.482908 8120 catalog_manager.cc:1347] Generated new cluster ID: 6619b8224fad4b31a37bd0610f90e7f5
I20260807 20:08:25.482967 8120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:25.492945 8120 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:25.493116 8120 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:25.495805 8120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446: Generated new TSK 0
I20260807 20:08:25.495997 8120 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:25.502844 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36531
--builtin_ntp_servers=127.3.15.148:40437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 20:08:25.587529 8130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:25.587778 8130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:25.587815 8130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:25.589032 8130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:25.589084 8130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:25.589128 8130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:25.590471 8130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:25.590741 8130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:25.591035 8130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:25.596911 8138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.596911 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.597046 8130 server_base.cc:1034] running on GCE node
W20260807 20:08:25.597213 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.597420 8130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:25.597687 8130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:25.598838 8130 hybrid_clock.cc:648] HybridClock initialized: now 1786133305598817 us; error 39 us; skew 500 ppm
I20260807 20:08:25.600150 8130 webserver.cc:460] Webserver started at http://127.3.15.129:34621/ using document root <none> and password file <none>
I20260807 20:08:25.600404 8130 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:25.600450 8130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:25.600607 8130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:25.601775 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "bd69a7ac9a1347cbbde2eae3410432fc"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "fedcbdc872ba0b09bec14712da308014"
server_key_iv: "fc6a2580b075561c1d899437d016fc64"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.602144 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "bd69a7ac9a1347cbbde2eae3410432fc"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "fedcbdc872ba0b09bec14712da308014"
server_key_iv: "fc6a2580b075561c1d899437d016fc64"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.603528 8130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:25.604321 8144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.604472 8130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:25.604564 8130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "bd69a7ac9a1347cbbde2eae3410432fc"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "fedcbdc872ba0b09bec14712da308014"
server_key_iv: "fc6a2580b075561c1d899437d016fc64"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.604647 8130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:25.608043 8130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:25.608325 8130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:25.608474 8130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:25.608726 8130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:25.609076 8130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:25.609112 8130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.609133 8130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:25.609175 8130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.615264 8130 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:46789
I20260807 20:08:25.615310 8264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:46789 every 8 connection(s)
I20260807 20:08:25.615672 8130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:25.618757 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8130
I20260807 20:08:25.618885 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:25.619066 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.616459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:53483 (local address 127.3.15.190:36531)
0807 20:08:25.616510 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.616513 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.616985 (+ 472us) server_negotiation.cc:407] Connection header received
0807 20:08:25.617365 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.617366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.617372 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.617430 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.618729 (+ 1299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.618731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.618732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.618738 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.618759 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.618764 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.618845 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.618971 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.618988 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:25.619135 3134 external_mini_cluster.cc:1428] Setting key d4f697e25890212394eb6d38f01aaa3e
I20260807 20:08:25.619287 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.616468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36531 (local address 127.3.15.129:53483)
0807 20:08:25.616681 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.616718 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.617273 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.617508 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.617511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.617520 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.618543 (+ 1023us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:25.618544 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.618575 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:25.618608 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.618865 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.618872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.618876 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.618976 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.619045 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":135,"threads_started":1}
I20260807 20:08:25.619917 8266 heartbeater.cc:347] Connected to a master server at 127.3.15.190:36531
I20260807 20:08:25.620026 8266 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:25.620195 8266 heartbeater.cc:511] Master 127.3.15.190:36531 requested a full tablet report, sending...
I20260807 20:08:25.620222 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36531
--builtin_ntp_servers=127.3.15.148:40437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 20:08:25.620744 8062 ts_manager.cc:194] Registered new tserver with Master: bd69a7ac9a1347cbbde2eae3410432fc (127.3.15.129:46789)
W20260807 20:08:25.706326 8269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:25.706537 8269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:25.706560 8269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:25.707828 8269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:25.707870 8269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:25.707903 8269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:25.709385 8269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:25.709626 8269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:25.709908 8269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:25.714985 8274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.714990 8275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.714993 8277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.715276 8269 server_base.cc:1034] running on GCE node
I20260807 20:08:25.715485 8269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:25.715739 8269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:25.716935 8269 hybrid_clock.cc:648] HybridClock initialized: now 1786133305716910 us; error 37 us; skew 500 ppm
I20260807 20:08:25.718240 8269 webserver.cc:460] Webserver started at http://127.3.15.130:40499/ using document root <none> and password file <none>
I20260807 20:08:25.718493 8269 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:25.718562 8269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:25.718708 8269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:25.719796 8269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "24caeb5c845f45898eb1ef99f24b5590"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "4977481094e5e7b06f05cd25014ec9f9"
server_key_iv: "328db03ce17ebfd2c923ecefb58dc7b6"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.720167 8269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "24caeb5c845f45898eb1ef99f24b5590"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "4977481094e5e7b06f05cd25014ec9f9"
server_key_iv: "328db03ce17ebfd2c923ecefb58dc7b6"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.721537 8269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:25.722245 8283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.722405 8269 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:25.722506 8269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "24caeb5c845f45898eb1ef99f24b5590"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "4977481094e5e7b06f05cd25014ec9f9"
server_key_iv: "328db03ce17ebfd2c923ecefb58dc7b6"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.722579 8269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:25.726634 8269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:25.726902 8269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:25.727064 8269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:25.727280 8269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:25.727594 8269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:25.727648 8269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.727689 8269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:25.727717 8269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.733377 8269 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:46199
I20260807 20:08:25.733422 8407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:46199 every 8 connection(s)
I20260807 20:08:25.733755 8269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:25.735914 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8269
I20260807 20:08:25.736006 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:25.736217 3134 external_mini_cluster.cc:1428] Setting key 635d623abecfcd9a452fe70f2b64e3d3
I20260807 20:08:25.736972 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.734482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:48891 (local address 127.3.15.190:36531)
0807 20:08:25.734563 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.734565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.734782 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:08:25.735258 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.735259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.735264 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.735319 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.736697 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.736699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.736699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.736704 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.736720 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.736725 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.736802 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.736885 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.736901 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 20:08:25.737190 8410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.734453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36531 (local address 127.3.15.130:48891)
0807 20:08:25.734666 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.734702 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.735171 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.735372 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.735375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.735384 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.736518 (+ 1134us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:25.736518 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.736560 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:25.736596 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.736807 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.736812 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.736816 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.736885 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.736951 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":109,"threads_started":1}
I20260807 20:08:25.737449 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36531
--builtin_ntp_servers=127.3.15.148:40437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 20:08:25.739225 8408 heartbeater.cc:347] Connected to a master server at 127.3.15.190:36531
I20260807 20:08:25.739319 8408 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:25.739486 8408 heartbeater.cc:511] Master 127.3.15.190:36531 requested a full tablet report, sending...
I20260807 20:08:25.739861 8062 ts_manager.cc:194] Registered new tserver with Master: 24caeb5c845f45898eb1ef99f24b5590 (127.3.15.130:46199)
W20260807 20:08:25.821749 8411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:25.821952 8411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:25.821975 8411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:25.823179 8411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:25.823220 8411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:25.823264 8411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:25.824630 8411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:36531
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:25.824890 8411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:25.825192 8411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:25.830355 8421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.830543 8411 server_base.cc:1034] running on GCE node
W20260807 20:08:25.830350 8419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:25.830446 8418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:25.830891 8411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:25.831169 8411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:25.832348 8411 hybrid_clock.cc:648] HybridClock initialized: now 1786133305832323 us; error 38 us; skew 500 ppm
I20260807 20:08:25.833763 8411 webserver.cc:460] Webserver started at http://127.3.15.131:33747/ using document root <none> and password file <none>
I20260807 20:08:25.834022 8411 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:25.834095 8411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:25.834266 8411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:25.835407 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "4a408247faa04d1ebb0e37b799aff71b"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "e47afa2b66c4e1ec46dacc11d17dd004"
server_key_iv: "88bebb89a36b28c03f212e14a61983b6"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.835815 8411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "4a408247faa04d1ebb0e37b799aff71b"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "e47afa2b66c4e1ec46dacc11d17dd004"
server_key_iv: "88bebb89a36b28c03f212e14a61983b6"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.837319 8411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:25.838068 8428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.838244 8411 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:25.838351 8411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "4a408247faa04d1ebb0e37b799aff71b"
format_stamp: "Formatted at 2026-08-07 20:08:25 on dist-test-slave-1x4g"
server_key: "e47afa2b66c4e1ec46dacc11d17dd004"
server_key_iv: "88bebb89a36b28c03f212e14a61983b6"
server_key_version: "encryptionkey@0"
I20260807 20:08:25.838431 8411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:25.841967 8411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:25.842242 8411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:25.842396 8411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:25.842609 8411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:25.842927 8411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:25.842980 8411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.843024 8411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:25.843050 8411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:25.849493 8411 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:42323
I20260807 20:08:25.849556 8555 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:42323 every 8 connection(s)
I20260807 20:08:25.849951 8411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:25.853009 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.850709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:44283 (local address 127.3.15.190:36531)
0807 20:08:25.850794 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.850797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.851001 (+ 204us) server_negotiation.cc:407] Connection header received
0807 20:08:25.851481 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.851482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.851487 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.851583 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.852749 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.852750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.852751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.852755 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.852772 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.852801 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.852850 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.852918 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.852933 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 20:08:25.853226 8558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.850679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:36531 (local address 127.3.15.131:44283)
0807 20:08:25.850886 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.850919 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.851366 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.851668 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.851673 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.851683 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.852584 (+ 901us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:25.852584 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.852612 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:25.852642 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.852847 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.852853 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.852856 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.852931 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.853004 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":113,"threads_started":1}
I20260807 20:08:25.853454 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8411
I20260807 20:08:25.853557 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:25.853745 8556 heartbeater.cc:347] Connected to a master server at 127.3.15.190:36531
I20260807 20:08:25.853773 3134 external_mini_cluster.cc:1428] Setting key ce50d0014ceecbc66cf0e63bfb57fa2e
I20260807 20:08:25.853852 8556 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:25.854007 8556 heartbeater.cc:511] Master 127.3.15.190:36531 requested a full tablet report, sending...
I20260807 20:08:25.854364 8062 ts_manager.cc:194] Registered new tserver with Master: 4a408247faa04d1ebb0e37b799aff71b (127.3.15.131:42323)
I20260807 20:08:25.855122 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:08:25.857611 8110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.856458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.3.15.190:36531)
0807 20:08:25.856647 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.856650 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.856725 (+ 75us) server_negotiation.cc:407] Connection header received
0807 20:08:25.856814 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.856815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.856820 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.856869 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.857396 (+ 527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.857397 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.857398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.857402 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.857415 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.857419 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.857469 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.857527 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.857541 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260807 20:08:25.859149 8062 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:25.860996 8062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:25.864925 8564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.863075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:46199 (local address 127.0.0.1:56010)
0807 20:08:25.863283 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.863329 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.863388 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.863956 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.863957 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.863960 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.864439 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:25.864440 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.864455 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:25.864478 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.864728 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.864734 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.864738 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.864841 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.864856 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":72,"threads_started":1}
I20260807 20:08:25.864945 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.863269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56010 (local address 127.3.15.130:46199)
0807 20:08:25.863729 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.863733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.863742 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:25.863770 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.863772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.863780 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.864140 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.864562 (+ 422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.864569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.864570 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.864573 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.864607 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.864631 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.864734 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.864850 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.864867 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":86,"threads_started":1}
I20260807 20:08:25.865116 8569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.863117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46789 (local address 127.0.0.1:58140)
0807 20:08:25.863455 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.863464 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.863505 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.863712 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.863719 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.863725 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.864599 (+ 874us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:25.864600 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.864608 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:25.864623 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.864942 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.864943 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.864944 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.865004 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.865022 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":90,"threads_started":1}
I20260807 20:08:25.865406 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.863240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58140 (local address 127.3.15.129:46789)
0807 20:08:25.863555 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.863560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.863572 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:08:25.863605 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.863607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.863618 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.863709 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.864729 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.864738 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.864738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.864744 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.864831 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.864859 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.864947 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.865287 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.865307 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":100,"threads_started":1}
I20260807 20:08:25.865496 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.863297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42323 (local address 127.0.0.1:46686)
0807 20:08:25.863559 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:25.863565 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:25.863608 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:25.864119 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:25.864120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:25.864123 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:25.865139 (+ 1016us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:25.865139 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:25.865146 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:25.865158 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:25.865386 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:25.865386 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:25.865387 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:08:25.865427 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 20:08:25.865439 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":73,"threads_started":1}
I20260807 20:08:25.865753 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:25.863488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46686 (local address 127.3.15.131:42323)
0807 20:08:25.863691 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:25.863695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:25.863706 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:25.863733 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:25.863735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:25.863744 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:25.863818 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:25.865239 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:25.865246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:25.865247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:25.865251 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:25.865287 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:25.865310 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:25.865362 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:25.865648 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 20:08:25.865666 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260807 20:08:25.865686 8337 tablet_service.cc:1467] Processing CreateTablet for tablet 91f9bbbc940646d8a583c1a140a0af92 (DEFAULT_TABLE table=test-table [id=a65e27ddcf0f4cfcaa8bc59f4c3550e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:25.866127 8337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f9bbbc940646d8a583c1a140a0af92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.866062 8197 tablet_service.cc:1467] Processing CreateTablet for tablet 91f9bbbc940646d8a583c1a140a0af92 (DEFAULT_TABLE table=test-table [id=a65e27ddcf0f4cfcaa8bc59f4c3550e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:25.866422 8197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f9bbbc940646d8a583c1a140a0af92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.868678 8571 tablet_bootstrap.cc:492] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Bootstrap starting.
I20260807 20:08:25.869143 8572 tablet_bootstrap.cc:492] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: Bootstrap starting.
I20260807 20:08:25.869204 8484 tablet_service.cc:1467] Processing CreateTablet for tablet 91f9bbbc940646d8a583c1a140a0af92 (DEFAULT_TABLE table=test-table [id=a65e27ddcf0f4cfcaa8bc59f4c3550e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:25.869390 8571 tablet_bootstrap.cc:654] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.869560 8484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91f9bbbc940646d8a583c1a140a0af92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:25.869747 8571 log.cc:824] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.869868 8572 tablet_bootstrap.cc:654] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.870221 8572 log.cc:824] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.870590 8571 tablet_bootstrap.cc:492] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: No bootstrap required, opened a new log
I20260807 20:08:25.870679 8571 ts_tablet_manager.cc:1397] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:08:25.871004 8572 tablet_bootstrap.cc:492] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: No bootstrap required, opened a new log
I20260807 20:08:25.871090 8572 ts_tablet_manager.cc:1397] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:25.872915 8576 tablet_bootstrap.cc:492] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Bootstrap starting.
I20260807 20:08:25.873190 8572 raft_consensus.cc:348] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:25.873401 8572 raft_consensus.cc:374] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.873452 8572 raft_consensus.cc:729] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd69a7ac9a1347cbbde2eae3410432fc, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.873601 8572 consensus_queue.cc:260] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:25.873692 8576 tablet_bootstrap.cc:654] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:25.873590 8571 raft_consensus.cc:348] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:25.873751 8571 raft_consensus.cc:374] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.873795 8571 raft_consensus.cc:729] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24caeb5c845f45898eb1ef99f24b5590, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.873935 8572 ts_tablet_manager.cc:1428] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:08:25.873922 8571 consensus_queue.cc:260] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:25.874037 8576 log.cc:824] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:25.874087 8266 heartbeater.cc:503] Master 127.3.15.190:36531 was elected leader, sending a full tablet report...
I20260807 20:08:25.874511 8571 ts_tablet_manager.cc:1428] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 20:08:25.874822 8576 tablet_bootstrap.cc:492] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: No bootstrap required, opened a new log
I20260807 20:08:25.874907 8576 ts_tablet_manager.cc:1397] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:25.876911 8408 heartbeater.cc:503] Master 127.3.15.190:36531 was elected leader, sending a full tablet report...
I20260807 20:08:25.877190 8576 raft_consensus.cc:348] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:25.877331 8576 raft_consensus.cc:374] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:25.877385 8576 raft_consensus.cc:729] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a408247faa04d1ebb0e37b799aff71b, State: Initialized, Role: FOLLOWER
I20260807 20:08:25.877517 8576 consensus_queue.cc:260] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:25.877791 8576 ts_tablet_manager.cc:1428] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 20:08:25.877820 8556 heartbeater.cc:503] Master 127.3.15.190:36531 was elected leader, sending a full tablet report...
W20260807 20:08:25.984284 8409 tablet.cc:2369] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:08:26.100556 8557 tablet.cc:2369] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:08:26.116456 8267 tablet.cc:2369] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:26.142571 8578 raft_consensus.cc:482] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:26.142735 8578 raft_consensus.cc:504] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:26.143142 8578 leader_election.cc:288] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24caeb5c845f45898eb1ef99f24b5590 (127.3.15.130:46199), 4a408247faa04d1ebb0e37b799aff71b (127.3.15.131:42323)
I20260807 20:08:26.144747 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.143453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:40851 (local address 127.3.15.130:46199)
0807 20:08:26.143525 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.143528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.143607 (+ 79us) server_negotiation.cc:407] Connection header received
0807 20:08:26.143746 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.143747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.143753 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.143820 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:26.144467 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:26.144469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:26.144469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:26.144473 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:26.144505 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:26.144511 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:26.144582 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:26.144652 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 20:08:26.144665 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:26.144766 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.143362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:46199 (local address 127.3.15.129:40851)
0807 20:08:26.143511 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:26.143526 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:26.143619 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:26.143830 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:26.143831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:26.143834 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:26.144340 (+ 506us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:26.144341 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:26.144349 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:26.144366 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:26.144579 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:26.144579 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:26.144580 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:08:26.144650 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:08:26.144664 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":73,"threads_started":1}
I20260807 20:08:26.144944 8570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.143676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:41561 (local address 127.3.15.131:42323)
0807 20:08:26.143741 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.143743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.143752 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:26.143800 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.143801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.143806 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.143865 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:26.144639 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:26.144640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:26.144641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:26.144646 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:26.144665 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:26.144670 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:26.144727 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:26.144834 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 20:08:26.144849 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:08:26.144979 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.143563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42323 (local address 127.3.15.129:41561)
0807 20:08:26.143668 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:26.143677 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:26.143727 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:26.143875 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:26.143876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:26.143879 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:26.144532 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:26.144533 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:26.144540 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:26.144553 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:26.144732 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:26.144733 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:26.144733 (+ 0us) client_negotiation.cc:769] Sending connection context
0807 20:08:26.144829 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 20:08:26.144843 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":40,"threads_started":1}
I20260807 20:08:26.145460 8507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f9bbbc940646d8a583c1a140a0af92" candidate_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a408247faa04d1ebb0e37b799aff71b" is_pre_election: true
I20260807 20:08:26.145465 8358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f9bbbc940646d8a583c1a140a0af92" candidate_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24caeb5c845f45898eb1ef99f24b5590" is_pre_election: true
I20260807 20:08:26.145620 8507 raft_consensus.cc:2454] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd69a7ac9a1347cbbde2eae3410432fc in term 0.
I20260807 20:08:26.145635 8358 raft_consensus.cc:2454] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd69a7ac9a1347cbbde2eae3410432fc in term 0.
I20260807 20:08:26.145874 8147 leader_election.cc:302] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24caeb5c845f45898eb1ef99f24b5590, bd69a7ac9a1347cbbde2eae3410432fc; no voters:
I20260807 20:08:26.145993 8578 raft_consensus.cc:2781] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:26.146052 8578 raft_consensus.cc:482] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:26.146085 8578 raft_consensus.cc:3037] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:26.146843 8578 raft_consensus.cc:504] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:26.146991 8578 leader_election.cc:288] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [CANDIDATE]: Term 1 election: Requested vote from peers 24caeb5c845f45898eb1ef99f24b5590 (127.3.15.130:46199), 4a408247faa04d1ebb0e37b799aff71b (127.3.15.131:42323)
I20260807 20:08:26.147207 8358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f9bbbc940646d8a583c1a140a0af92" candidate_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24caeb5c845f45898eb1ef99f24b5590"
I20260807 20:08:26.147194 8507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91f9bbbc940646d8a583c1a140a0af92" candidate_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a408247faa04d1ebb0e37b799aff71b"
I20260807 20:08:26.147291 8358 raft_consensus.cc:3037] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:26.147292 8507 raft_consensus.cc:3037] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:26.148234 8507 raft_consensus.cc:2454] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd69a7ac9a1347cbbde2eae3410432fc in term 1.
I20260807 20:08:26.148233 8358 raft_consensus.cc:2454] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd69a7ac9a1347cbbde2eae3410432fc in term 1.
I20260807 20:08:26.148396 8147 leader_election.cc:302] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a408247faa04d1ebb0e37b799aff71b, bd69a7ac9a1347cbbde2eae3410432fc; no voters:
I20260807 20:08:26.148502 8578 raft_consensus.cc:2781] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:26.148634 8578 raft_consensus.cc:686] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [term 1 LEADER]: Becoming Leader. State: Replica: bd69a7ac9a1347cbbde2eae3410432fc, State: Running, Role: LEADER
I20260807 20:08:26.148739 8578 consensus_queue.cc:237] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } }
I20260807 20:08:26.149637 8062 catalog_manager.cc:5495] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc reported cstate change: term changed from 0 to 1, leader changed from <none> to bd69a7ac9a1347cbbde2eae3410432fc (127.3.15.129). New cstate: current_term: 1 leader_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd69a7ac9a1347cbbde2eae3410432fc" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 46789 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:26.155164 8567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.154180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.3.15.129:46789)
0807 20:08:26.154233 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.154235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.154242 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:26.154313 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.154314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.154325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.154374 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:26.154953 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:26.154954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:26.154954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:26.154958 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:26.154973 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:26.154976 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:26.155035 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:26.155069 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 20:08:26.155079 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:08:26.157034 8358 raft_consensus.cc:1261] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 1 FOLLOWER]: Refusing update from remote peer bd69a7ac9a1347cbbde2eae3410432fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:26.157034 8507 raft_consensus.cc:1261] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 1 FOLLOWER]: Refusing update from remote peer bd69a7ac9a1347cbbde2eae3410432fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:26.157444 8585 consensus_queue.cc:1035] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "24caeb5c845f45898eb1ef99f24b5590" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:26.157565 8578 consensus_queue.cc:1035] T 91f9bbbc940646d8a583c1a140a0af92 P bd69a7ac9a1347cbbde2eae3410432fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a408247faa04d1ebb0e37b799aff71b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42323 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:26.162717 8062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:26.162833 8062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:26.163617 8062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a0f740d054e461181a26c7e41856446: Sending DeleteTablet for 3 replicas of tablet 91f9bbbc940646d8a583c1a140a0af92
I20260807 20:08:26.163915 8337 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f9bbbc940646d8a583c1a140a0af92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:26 UTC) from {username='slave'} at 127.0.0.1:56010
I20260807 20:08:26.163982 8484 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f9bbbc940646d8a583c1a140a0af92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:26 UTC) from {username='slave'} at 127.0.0.1:46686
I20260807 20:08:26.164223 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8130
I20260807 20:08:26.164238 8599 tablet_replica.cc:330] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: stopping tablet replica
I20260807 20:08:26.164268 8197 tablet_service.cc:1514] Processing DeleteTablet for tablet 91f9bbbc940646d8a583c1a140a0af92 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:26 UTC) from {username='slave'} at 127.0.0.1:58140
I20260807 20:08:26.165817 8599 raft_consensus.cc:2229] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:26.165925 8599 raft_consensus.cc:2258] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:26.166071 8600 tablet_replica.cc:330] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: stopping tablet replica
I20260807 20:08:26.166162 8600 raft_consensus.cc:2229] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:26.166255 8600 raft_consensus.cc:2258] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:26.166392 8599 ts_tablet_manager.cc:1905] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:26.166697 8600 ts_tablet_manager.cc:1905] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:26.168092 8599 ts_tablet_manager.cc:1918] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:26.168186 8599 log.cc:1196] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/91f9bbbc940646d8a583c1a140a0af92
I20260807 20:08:26.168359 8600 ts_tablet_manager.cc:1918] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:26.168434 8600 log.cc:1196] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/91f9bbbc940646d8a583c1a140a0af92
I20260807 20:08:26.168462 8599 ts_tablet_manager.cc:1939] T 91f9bbbc940646d8a583c1a140a0af92 P 24caeb5c845f45898eb1ef99f24b5590: Deleting consensus metadata
I20260807 20:08:26.168706 8600 ts_tablet_manager.cc:1939] T 91f9bbbc940646d8a583c1a140a0af92 P 4a408247faa04d1ebb0e37b799aff71b: Deleting consensus metadata
I20260807 20:08:26.172077 8047 catalog_manager.cc:4841] TS 4a408247faa04d1ebb0e37b799aff71b (127.3.15.131:42323): tablet 91f9bbbc940646d8a583c1a140a0af92 (table test-table [id=a65e27ddcf0f4cfcaa8bc59f4c3550e0]) successfully deleted
I20260807 20:08:26.172209 8047 catalog_manager.cc:4841] TS 24caeb5c845f45898eb1ef99f24b5590 (127.3.15.130:46199): tablet 91f9bbbc940646d8a583c1a140a0af92 (table test-table [id=a65e27ddcf0f4cfcaa8bc59f4c3550e0]) successfully deleted
W20260807 20:08:26.172363 8046 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:46789 (error 108)
I20260807 20:08:26.172672 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8269
I20260807 20:08:26.173317 8566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.172716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:46789 (local address 127.0.0.1:58156)
0807 20:08:26.173228 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:26.173278 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:46789: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487}
W20260807 20:08:26.173702 8046 catalog_manager.cc:4584] TS bd69a7ac9a1347cbbde2eae3410432fc (127.3.15.129:46789): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91f9bbbc940646d8a583c1a140a0af92: Network error: Client connection negotiation failed: client connection to 127.3.15.129:46789: connect: Connection refused (error 111)
I20260807 20:08:26.182300 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8411
I20260807 20:08:26.187343 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8029
2026-08-07T20:08:26Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (857 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 20:08:26.209129 3134 mini_oidc.cc:152] Starting JWKS server
I20260807 20:08:26.210338 3134 webserver.cc:460] Webserver started at https://127.0.0.1:37877/ using document root <none> and password file <none>
I20260807 20:08:26.210455 3134 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 20:08:26.210660 3134 webserver.cc:460] Webserver started at http://127.0.0.1:38695/ using document root <none> and password file <none>
2026-08-07T20:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:26Z Disabled control of system clock
I20260807 20:08:26.219667 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35351
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:41219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:35351
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37877/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/testchainca.cert with env {}
W20260807 20:08:26.302861 8608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:26.303072 8608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:26.303094 8608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:26.304255 8608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:26.304294 8608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:26.304307 8608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:26.304319 8608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:26.304332 8608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:08:26.304344 8608 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:08:26.304355 8608 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37877/jwks.json
I20260807 20:08:26.305841 8608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:41219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:35351
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37877/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:26.306048 8608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:26.306202 8608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:26.311058 8616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:26.311089 8613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.311137 8608 server_base.cc:1034] running on GCE node
W20260807 20:08:26.311090 8614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.311565 8608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:26.311857 8608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:26.313042 8608 hybrid_clock.cc:648] HybridClock initialized: now 1786133306313027 us; error 40 us; skew 500 ppm
I20260807 20:08:26.318964 8608 webserver.cc:460] Webserver started at http://127.3.15.190:41467/ using document root <none> and password file <none>
I20260807 20:08:26.319242 8608 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:26.319311 8608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:26.319463 8608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:26.320549 8608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "d18767ced0a94de88e8f974d63ec69c1"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "37143655944b25262393b75a9a436e5e"
server_key_iv: "95f5f3079563b578b6938484d1236ede"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.320931 8608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "d18767ced0a94de88e8f974d63ec69c1"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "37143655944b25262393b75a9a436e5e"
server_key_iv: "95f5f3079563b578b6938484d1236ede"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.322283 8608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:26.323024 8622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:26.323197 8608 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:26.323302 8608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "d18767ced0a94de88e8f974d63ec69c1"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "37143655944b25262393b75a9a436e5e"
server_key_iv: "95f5f3079563b578b6938484d1236ede"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.323374 8608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:26.339145 8608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:26.339473 8608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:26.339622 8608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:26.343729 8608 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:35351
I20260807 20:08:26.343763 8679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:35351 every 8 connection(s)
I20260807 20:08:26.344121 8608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:26.345379 8680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:26.347702 8680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1: Bootstrap starting.
I20260807 20:08:26.348311 8680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:26.348618 8680 log.cc:824] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:26.349330 8680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1: No bootstrap required, opened a new log
I20260807 20:08:26.350920 8680 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18767ced0a94de88e8f974d63ec69c1" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35351 } }
I20260807 20:08:26.351059 8680 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:26.351081 8680 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18767ced0a94de88e8f974d63ec69c1, State: Initialized, Role: FOLLOWER
I20260807 20:08:26.351256 8680 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18767ced0a94de88e8f974d63ec69c1" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35351 } }
I20260807 20:08:26.351336 8680 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:26.351387 8680 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:26.351440 8680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:26.351953 8680 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18767ced0a94de88e8f974d63ec69c1" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35351 } }
I20260807 20:08:26.352065 8680 leader_election.cc:302] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18767ced0a94de88e8f974d63ec69c1; no voters:
I20260807 20:08:26.352244 8680 leader_election.cc:288] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:26.352290 8683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:26.352458 8683 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [term 1 LEADER]: Becoming Leader. State: Replica: d18767ced0a94de88e8f974d63ec69c1, State: Running, Role: LEADER
I20260807 20:08:26.352509 8680 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:26.352555 8683 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18767ced0a94de88e8f974d63ec69c1" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35351 } }
I20260807 20:08:26.353016 8686 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18767ced0a94de88e8f974d63ec69c1. Latest consensus state: current_term: 1 leader_uuid: "d18767ced0a94de88e8f974d63ec69c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18767ced0a94de88e8f974d63ec69c1" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35351 } } }
I20260807 20:08:26.353108 8686 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:26.353113 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18767ced0a94de88e8f974d63ec69c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18767ced0a94de88e8f974d63ec69c1" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35351 } } }
I20260807 20:08:26.353287 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:26.353374 8693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:26.353914 8693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:26.354348 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8608
I20260807 20:08:26.354471 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:26.354674 3134 external_mini_cluster.cc:1428] Setting key 1d3e1c7fbe610f0c09b99d70b0694474
I20260807 20:08:26.355432 8693 catalog_manager.cc:1347] Generated new cluster ID: 06790febe9a4451d91d4d0c4413ae37a
I20260807 20:08:26.355485 8693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:26.362172 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.355313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58580 (local address 127.3.15.190:35351)
0807 20:08:26.356918 (+ 1605us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.356926 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.356969 (+ 43us) server_negotiation.cc:407] Connection header received
0807 20:08:26.357091 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.357099 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.357188 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.357339 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:26.357566 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.358936 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.359990 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.360646 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.360803 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.361482 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:26.361486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:26.361486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:26.361690 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:26.361732 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:26.361746 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:26.361816 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:26.361879 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 20:08:26.361971 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1530,"thread_start_us":76,"threads_started":1}
I20260807 20:08:26.363097 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:08:26.374429 8693 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:26.375100 8693 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:26.377846 8693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18767ced0a94de88e8f974d63ec69c1: Generated new TSK 0
I20260807 20:08:26.378023 8693 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 20:08:26.399626 8712 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 20:08:26.399807 8712 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 20:08:26.400064 8705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.384284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58586 (local address 127.3.15.190:35351)
0807 20:08:26.384351 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.384354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.384501 (+ 147us) server_negotiation.cc:407] Connection header received
0807 20:08:26.384629 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.384630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.384641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.384706 (+ 65us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:08:26.384898 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.385379 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.386539 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.386726 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.386793 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.387032 (+ 239us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:08:26.399771 (+ 12739us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37877/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 20:08:26.399987 (+ 216us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58586: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37877/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 20:08:26.400134 8705 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58586: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37877/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 20:08:26.400637 8711 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:26.384205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35351 (local address 127.0.0.1:58586)
0807 20:08:26.384439 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:26.384452 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:26.384524 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:26.384734 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:26.384735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:26.384738 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:08:26.384830 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:26.384833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.385462 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.385463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:26.386446 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:26.386449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.386813 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.386814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:26.386882 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.386904 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:08:26.400013 (+ 13109us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37877/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 20:08:26.400148 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.15.190:35351: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37877/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":122,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.15.190:35351: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37877/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 20:08:26.401124 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8608
2026-08-07T20:08:26Z chronyd exiting
I20260807 20:08:26.439767 3134 test_util.cc:175] -----------------------------------------------
I20260807 20:08:26.439870 3134 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786133281907902-3134-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 20:08:26.440740 3134 mini_oidc.cc:152] Starting JWKS server
I20260807 20:08:26.441478 3134 webserver.cc:460] Webserver started at https://127.0.0.1:44677/ using document root <none> and password file <none>
I20260807 20:08:26.441545 3134 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 20:08:26.441705 3134 webserver.cc:460] Webserver started at http://127.0.0.1:34427/ using document root <none> and password file <none>
2026-08-07T20:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:26Z Disabled control of system clock
I20260807 20:08:26.450637 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:41309
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:46821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:41309
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44677/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/testchainca.cert with env {}
W20260807 20:08:26.533838 8718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:26.534063 8718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:26.534086 8718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:26.535326 8718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:26.535372 8718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:26.535388 8718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:26.535403 8718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:26.535418 8718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:08:26.535430 8718 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:08:26.535445 8718 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44677/jwks.json
I20260807 20:08:26.537055 8718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:41309
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:41309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44677/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:26.537266 8718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:26.537432 8718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:26.542325 8724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:26.542327 8727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:26.542560 8725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.542843 8718 server_base.cc:1034] running on GCE node
I20260807 20:08:26.543118 8718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:26.543434 8718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:26.544595 8718 hybrid_clock.cc:648] HybridClock initialized: now 1786133306544588 us; error 35 us; skew 500 ppm
I20260807 20:08:26.545843 8718 webserver.cc:460] Webserver started at http://127.3.15.190:42307/ using document root <none> and password file <none>
I20260807 20:08:26.546103 8718 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:26.546168 8718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:26.546314 8718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:26.547384 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "3ed66e1f04d944f2a904b319a4fefd1f"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "ae349599b275cde7c165fcef5dfe7f89"
server_key_iv: "6bc70d6150dfea2f5e899e2029e8838a"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.547749 8718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "3ed66e1f04d944f2a904b319a4fefd1f"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "ae349599b275cde7c165fcef5dfe7f89"
server_key_iv: "6bc70d6150dfea2f5e899e2029e8838a"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.549085 8718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:26.549793 8734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:26.549969 8718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:26.550074 8718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "3ed66e1f04d944f2a904b319a4fefd1f"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "ae349599b275cde7c165fcef5dfe7f89"
server_key_iv: "6bc70d6150dfea2f5e899e2029e8838a"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.550145 8718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:26.560595 8718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:26.560969 8718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:26.561121 8718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:26.564961 8718 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:41309
I20260807 20:08:26.565007 8789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:41309 every 8 connection(s)
I20260807 20:08:26.565344 8718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:26.566330 8790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:26.568601 8790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f: Bootstrap starting.
I20260807 20:08:26.569244 8790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:26.569523 8790 log.cc:824] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:26.570230 8790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f: No bootstrap required, opened a new log
I20260807 20:08:26.571789 8790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41309 } }
I20260807 20:08:26.571928 8790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:26.571950 8790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ed66e1f04d944f2a904b319a4fefd1f, State: Initialized, Role: FOLLOWER
I20260807 20:08:26.572103 8790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41309 } }
I20260807 20:08:26.572171 8790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:26.572193 8790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:26.572243 8790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:26.572849 8790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41309 } }
I20260807 20:08:26.572939 8790 leader_election.cc:302] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ed66e1f04d944f2a904b319a4fefd1f; no voters:
I20260807 20:08:26.573104 8790 leader_election.cc:288] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:26.573187 8794 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:26.573374 8794 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [term 1 LEADER]: Becoming Leader. State: Replica: 3ed66e1f04d944f2a904b319a4fefd1f, State: Running, Role: LEADER
I20260807 20:08:26.573400 8790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:26.573475 8794 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41309 } }
I20260807 20:08:26.573889 8795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41309 } } }
I20260807 20:08:26.573979 8795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:26.574294 8803 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:26.574270 8796 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ed66e1f04d944f2a904b319a4fefd1f. Latest consensus state: current_term: 1 leader_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ed66e1f04d944f2a904b319a4fefd1f" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41309 } } }
I20260807 20:08:26.574544 8796 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:26.574853 8803 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:26.575115 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8718
I20260807 20:08:26.575225 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:26.575418 3134 external_mini_cluster.cc:1428] Setting key 841ebfb3985fe7cdeb4fd6c577d455a3
I20260807 20:08:26.576619 8803 catalog_manager.cc:1347] Generated new cluster ID: 26c5789be3c34ae195ee2af6aa9d8b98
I20260807 20:08:26.576673 8803 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:26.581988 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.575868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.3.15.190:41309)
0807 20:08:26.576055 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.576064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.576357 (+ 293us) server_negotiation.cc:407] Connection header received
0807 20:08:26.576494 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.576501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.576572 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.576728 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:26.577084 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.578471 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.579584 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.580188 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.580294 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.581028 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:26.581032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:26.581032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:26.581243 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:26.581289 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:26.581304 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:26.581370 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:26.581495 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 20:08:26.581665 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260807 20:08:26.582893 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:08:26.584597 8803 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:26.585132 8803 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:26.590530 8803 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ed66e1f04d944f2a904b319a4fefd1f: Generated new TSK 0
I20260807 20:08:26.590696 8803 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 20:08:26.614254 8822 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 20:08:26.614336 8822 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 20:08:26.614676 8814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.600399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36990 (local address 127.3.15.190:41309)
0807 20:08:26.600460 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.600462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.600547 (+ 85us) server_negotiation.cc:407] Connection header received
0807 20:08:26.600599 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.600600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.600613 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.600683 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:08:26.600871 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.601643 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.602398 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.602764 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.602836 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.602969 (+ 133us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:08:26.614458 (+ 11489us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 20:08:26.614606 (+ 148us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 20:08:26.614733 8814 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36990: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 20:08:26.614743 8821 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:26.600314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:41309 (local address 127.0.0.1:36990)
0807 20:08:26.600477 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:26.600488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:26.600546 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:26.600697 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:26.600698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:26.600700 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:08:26.600749 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:26.600751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.601735 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.601736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:26.602322 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:26.602324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.602839 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.602839 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:26.602880 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.602885 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:08:26.614557 (+ 11672us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 20:08:26.614650 (+ 93us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.15.190:41309: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44677/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":96,"threads_started":1}
I20260807 20:08:26.615166 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8718
2026-08-07T20:08:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (213 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 20:08:26.654779 3134 mini_oidc.cc:152] Starting JWKS server
I20260807 20:08:26.655448 3134 webserver.cc:460] Webserver started at https://127.0.0.1:37597/ using document root <none> and password file <none>
I20260807 20:08:26.655489 3134 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 20:08:26.655601 3134 webserver.cc:460] Webserver started at http://127.0.0.1:46271/ using document root <none> and password file <none>
2026-08-07T20:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:26Z Disabled control of system clock
I20260807 20:08:26.664882 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39935
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:38373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39935
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37597/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 20:08:26.746863 8828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:26.747071 8828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:26.747092 8828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:26.748167 8828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:26.748202 8828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:26.748214 8828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:26.748227 8828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:26.748239 8828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:08:26.748252 8828 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:08:26.748270 8828 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37597/jwks.json
I20260807 20:08:26.749580 8828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:38373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37597/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:26.749780 8828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:26.750020 8828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:26.754846 8835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.754864 8828 server_base.cc:1034] running on GCE node
W20260807 20:08:26.754866 8834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:26.754833 8838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.755296 8828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:26.755633 8828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:26.756825 8828 hybrid_clock.cc:648] HybridClock initialized: now 1786133306756846 us; error 109 us; skew 500 ppm
I20260807 20:08:26.758195 8828 webserver.cc:460] Webserver started at http://127.3.15.190:37921/ using document root <none> and password file <none>
I20260807 20:08:26.758509 8828 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:26.758590 8828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:26.758749 8828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:26.759943 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "1e4e6255cd174841bbb0056239c9a012"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "4c2b7c97c2abd8703e88cacd4263389a"
server_key_iv: "2c785fa571efdbce306ff76f10141608"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.760422 8828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "1e4e6255cd174841bbb0056239c9a012"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "4c2b7c97c2abd8703e88cacd4263389a"
server_key_iv: "2c785fa571efdbce306ff76f10141608"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.761973 8828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 20:08:26.762746 8844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:26.762925 8828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:26.763024 8828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "1e4e6255cd174841bbb0056239c9a012"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "4c2b7c97c2abd8703e88cacd4263389a"
server_key_iv: "2c785fa571efdbce306ff76f10141608"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.763108 8828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:26.785717 8828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:26.786062 8828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:26.786217 8828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:26.790356 8828 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39935
I20260807 20:08:26.790423 8900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39935 every 8 connection(s)
I20260807 20:08:26.790766 8828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:26.791460 8901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:26.794219 8901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012: Bootstrap starting.
I20260807 20:08:26.794761 8901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:26.795006 8901 log.cc:824] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:26.795689 8901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012: No bootstrap required, opened a new log
I20260807 20:08:26.797302 8901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4e6255cd174841bbb0056239c9a012" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39935 } }
I20260807 20:08:26.797448 8901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:26.797473 8901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e4e6255cd174841bbb0056239c9a012, State: Initialized, Role: FOLLOWER
I20260807 20:08:26.797622 8901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4e6255cd174841bbb0056239c9a012" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39935 } }
I20260807 20:08:26.797708 8901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:26.797731 8901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:26.797763 8901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:26.798278 8901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4e6255cd174841bbb0056239c9a012" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39935 } }
I20260807 20:08:26.798365 8901 leader_election.cc:302] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e4e6255cd174841bbb0056239c9a012; no voters:
I20260807 20:08:26.798503 8901 leader_election.cc:288] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:26.798571 8904 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:26.798741 8904 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [term 1 LEADER]: Becoming Leader. State: Replica: 1e4e6255cd174841bbb0056239c9a012, State: Running, Role: LEADER
I20260807 20:08:26.798792 8901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:26.798844 8904 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4e6255cd174841bbb0056239c9a012" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39935 } }
I20260807 20:08:26.799127 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8828
I20260807 20:08:26.799239 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:26.799227 8906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e4e6255cd174841bbb0056239c9a012. Latest consensus state: current_term: 1 leader_uuid: "1e4e6255cd174841bbb0056239c9a012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4e6255cd174841bbb0056239c9a012" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39935 } } }
I20260807 20:08:26.799306 8906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:26.799345 8905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e4e6255cd174841bbb0056239c9a012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e4e6255cd174841bbb0056239c9a012" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39935 } } }
I20260807 20:08:26.799400 8905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:26.799472 3134 external_mini_cluster.cc:1428] Setting key 660156bde881f25a14a2e0e7684912b0
I20260807 20:08:26.800009 8912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:26.800681 8912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:26.802342 8912 catalog_manager.cc:1347] Generated new cluster ID: 0463e19dc70f4b7c83cb5e98dd08ec8f
I20260807 20:08:26.802397 8912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:26.806118 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.800377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56878 (local address 127.3.15.190:39935)
0807 20:08:26.800609 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.800618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.800653 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:08:26.800832 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.800841 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.800915 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.801065 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:26.801319 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.802830 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.803931 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.804563 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.804692 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.805385 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:26.805389 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:26.805389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:26.805601 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:26.805646 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:26.805661 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:26.805725 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:26.805801 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 20:08:26.805881 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
I20260807 20:08:26.807051 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:08:26.815012 8912 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:26.815513 8912 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:26.822034 8912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e4e6255cd174841bbb0056239c9a012: Generated new TSK 0
I20260807 20:08:26.822212 8912 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:26.836911 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:26.830461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56882 (local address 127.3.15.190:39935)
0807 20:08:26.830504 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:26.830506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:26.832443 (+ 1937us) server_negotiation.cc:407] Connection header received
0807 20:08:26.832538 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:26.832539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:26.832553 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:26.832635 (+ 82us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:08:26.832846 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.833675 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.834884 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.835168 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.835250 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.835476 (+ 226us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:08:26.836640 (+ 1164us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 20:08:26.836824 (+ 184us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56882: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 20:08:26.836980 8921 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56882: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 20:08:26.839232 8933 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:26.830490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39935 (local address 127.0.0.1:56882)
0807 20:08:26.832350 (+ 1860us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:26.832364 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:26.832441 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:26.832648 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:26.832649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:26.832652 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:08:26.832729 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:26.832732 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.833783 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.833785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:26.834781 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:26.834784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:26.835259 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:26.835260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:26.835329 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:26.835336 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:08:26.837154 (+ 1818us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 20:08:26.839127 (+ 1973us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.15.190:39935: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":1829,"mutex_wait_us":1775,"thread_start_us":152,"threads_started":1}
I20260807 20:08:26.839596 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8828
E20260807 20:08:26.853053 8934 webserver.cc:533] Webserver: sslize failed:
E20260807 20:08:26.853152 8934 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T20:08:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T20:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:26Z Disabled control of system clock
I20260807 20:08:26.890828 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45277
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:45277
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/testchainca.cert with env {}
W20260807 20:08:26.974239 8939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:26.974490 8939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:26.974545 8939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:26.975699 8939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:26.975768 8939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:26.975793 8939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:26.975817 8939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:26.975849 8939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 20:08:26.975885 8939 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 20:08:26.975914 8939 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 20:08:26.977458 8939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:45277
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:26.977736 8939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:26.977977 8939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:26.982992 8944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:26.982961 8947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.983122 8939 server_base.cc:1034] running on GCE node
W20260807 20:08:26.982967 8945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:26.983467 8939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:26.983758 8939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:26.984927 8939 hybrid_clock.cc:648] HybridClock initialized: now 1786133306984906 us; error 36 us; skew 500 ppm
I20260807 20:08:26.986181 8939 webserver.cc:460] Webserver started at http://127.3.15.190:39231/ using document root <none> and password file <none>
I20260807 20:08:26.986397 8939 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:26.986455 8939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:26.986553 8939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:26.987543 8939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "55d4941e73484c9db6eff87e4159f6f6"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "57630ec3cbc9f34e2b2a14d7b2d5dd9d"
server_key_iv: "8dfae64858ec3f1f12c48a4c4b893977"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.987871 8939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "55d4941e73484c9db6eff87e4159f6f6"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "57630ec3cbc9f34e2b2a14d7b2d5dd9d"
server_key_iv: "8dfae64858ec3f1f12c48a4c4b893977"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.989204 8939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:26.989938 8953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:26.990128 8939 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:26.990216 8939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "55d4941e73484c9db6eff87e4159f6f6"
format_stamp: "Formatted at 2026-08-07 20:08:26 on dist-test-slave-1x4g"
server_key: "57630ec3cbc9f34e2b2a14d7b2d5dd9d"
server_key_iv: "8dfae64858ec3f1f12c48a4c4b893977"
server_key_version: "encryptionkey@0"
I20260807 20:08:26.990269 8939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:27.017354 8939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:27.017640 8939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:27.017750 8939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:27.021838 8939 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:45277
I20260807 20:08:27.021909 9012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:45277 every 8 connection(s)
I20260807 20:08:27.022265 8939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:27.023300 9013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:27.025487 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 8939
I20260807 20:08:27.025607 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:27.025821 3134 external_mini_cluster.cc:1428] Setting key 7d4924e9e1e3d96401003efd98fff7b7
I20260807 20:08:27.026014 9013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6: Bootstrap starting.
I20260807 20:08:27.026712 9013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:27.027006 9013 log.cc:824] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:27.027751 9013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6: No bootstrap required, opened a new log
I20260807 20:08:27.029557 9013 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d4941e73484c9db6eff87e4159f6f6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45277 } }
I20260807 20:08:27.029692 9013 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:27.029716 9013 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d4941e73484c9db6eff87e4159f6f6, State: Initialized, Role: FOLLOWER
I20260807 20:08:27.029858 9013 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d4941e73484c9db6eff87e4159f6f6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45277 } }
I20260807 20:08:27.029953 9013 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:27.029986 9013 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:27.030040 9013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:27.030704 9013 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d4941e73484c9db6eff87e4159f6f6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45277 } }
I20260807 20:08:27.030793 9013 leader_election.cc:302] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55d4941e73484c9db6eff87e4159f6f6; no voters:
I20260807 20:08:27.030984 9013 leader_election.cc:288] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:27.031003 9020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:27.031111 9020 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [term 1 LEADER]: Becoming Leader. State: Replica: 55d4941e73484c9db6eff87e4159f6f6, State: Running, Role: LEADER
I20260807 20:08:27.031224 9020 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d4941e73484c9db6eff87e4159f6f6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45277 } }
I20260807 20:08:27.031251 9013 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:27.031322 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:27.026359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.3.15.190:45277)
0807 20:08:27.026563 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:27.026571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:27.026597 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:08:27.027100 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:27.027110 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:27.027178 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:27.027322 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:27.027540 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:27.028485 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:27.029333 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:27.029788 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:27.029915 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:27.030562 (+ 647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:27.030566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:27.030566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:27.030733 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:27.030786 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:27.030802 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:27.030872 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:27.030950 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 20:08:27.031031 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260807 20:08:27.032161 9021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55d4941e73484c9db6eff87e4159f6f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d4941e73484c9db6eff87e4159f6f6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45277 } } }
I20260807 20:08:27.032253 9021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:27.032598 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 20:08:27.033164 9038 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:08:27.033221 9038 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:08:27.032212 9022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55d4941e73484c9db6eff87e4159f6f6. Latest consensus state: current_term: 1 leader_uuid: "55d4941e73484c9db6eff87e4159f6f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55d4941e73484c9db6eff87e4159f6f6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45277 } } }
I20260807 20:08:27.033365 9022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:27.033353 9033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:27.033660 9033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:27.035079 9033 catalog_manager.cc:1347] Generated new cluster ID: 0102101ca17147c6b8c6425f96d5edd2
I20260807 20:08:27.035147 9033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:27.043468 9033 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:27.043952 9033 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:27.051054 9033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55d4941e73484c9db6eff87e4159f6f6: Generated new TSK 0
I20260807 20:08:27.051225 9033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:28.302479 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:27.060262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.3.15.190:45277)
0807 20:08:27.060330 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:27.060332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:27.060433 (+ 101us) server_negotiation.cc:407] Connection header received
0807 20:08:27.060521 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:27.060521 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:27.060532 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:27.060599 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0807 20:08:27.060753 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:27.061308 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:27.062473 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:27.062679 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:27.062742 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:27.062953 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 20:08:28.302212 (+1239259us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 20:08:28.302390 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54442: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 20:08:28.302579 9018 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54442: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 20:08:28.302589 9049 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:27.060188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45277 (local address 127.0.0.1:54442)
0807 20:08:27.060367 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:27.060379 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:27.060447 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:27.060621 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:27.060622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:27.060624 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 20:08:27.060690 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:27.060693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:27.061404 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:27.061405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:27.062385 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:27.062388 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:27.062775 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:27.062776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:27.062844 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:27.062850 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 20:08:28.302367 (+1239517us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 20:08:28.302501 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.15.190:45277: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260807 20:08:28.302970 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 8939
2026-08-07T20:08:28Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1440 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (92 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:28 dist-test-slave-1x4g krb5kdc[9061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:28 dist-test-slave-1x4g krb5kdc[9061](info): set up 2 sockets
Aug 07 20:08:28 dist-test-slave-1x4g krb5kdc[9061](info): commencing operation
krb5kdc: starting...
W20260807 20:08:29.662228 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:29 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:29.787370 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35797
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:35797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:29.871806 9075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:29.872025 9075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:29.872054 9075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:29.873278 9075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:29.873317 9075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:29.873332 9075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:29.873381 9075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:29.873407 9075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:29.874833 9075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:35797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:29.875093 9075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:29.875319 9075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:29.880242 9080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:29.880242 9081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:29.880268 9083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:29.880438 9075 server_base.cc:1034] running on GCE node
I20260807 20:08:29.880714 9075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:29.881047 9075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:29.882215 9075 hybrid_clock.cc:648] HybridClock initialized: now 1786133309882197 us; error 37 us; skew 500 ppm
Aug 07 20:08:29 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133309, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:29.884878 9075 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:29.885335 9075 webserver.cc:460] Webserver started at http://127.3.15.190:41353/ using document root <none> and password file <none>
I20260807 20:08:29.885565 9075 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:29.885628 9075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:29.885774 9075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:29.886864 9075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9"
format_stamp: "Formatted at 2026-08-07 20:08:29 on dist-test-slave-1x4g"
server_key: "a11a78022461571b707508cb488a0aa2"
server_key_iv: "7e1b9a102054d78c8da266c1a2ea6a26"
server_key_version: "encryptionkey@0"
I20260807 20:08:29.887243 9075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9"
format_stamp: "Formatted at 2026-08-07 20:08:29 on dist-test-slave-1x4g"
server_key: "a11a78022461571b707508cb488a0aa2"
server_key_iv: "7e1b9a102054d78c8da266c1a2ea6a26"
server_key_version: "encryptionkey@0"
I20260807 20:08:29.888616 9075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:29.889505 9091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:29.889698 9075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:29.889798 9075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9"
format_stamp: "Formatted at 2026-08-07 20:08:29 on dist-test-slave-1x4g"
server_key: "a11a78022461571b707508cb488a0aa2"
server_key_iv: "7e1b9a102054d78c8da266c1a2ea6a26"
server_key_version: "encryptionkey@0"
I20260807 20:08:29.889878 9075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:29.898195 9075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:29.900152 9075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:29.900309 9075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:29.904417 9075 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:35797
I20260807 20:08:29.904466 9149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:35797 every 8 connection(s)
I20260807 20:08:29.904906 9075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:29.905866 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:29.908115 9150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9: Bootstrap starting.
I20260807 20:08:29.908676 9150 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:29.909008 9150 log.cc:824] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:29.909688 9150 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9: No bootstrap required, opened a new log
I20260807 20:08:29.911253 9150 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35797 } }
I20260807 20:08:29.911396 9150 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:29.911420 9150 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba993fccc3734c6bab5d17ffd5d6cdf9, State: Initialized, Role: FOLLOWER
I20260807 20:08:29.911564 9150 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35797 } }
I20260807 20:08:29.911641 9150 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:29.911664 9150 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:29.911692 9150 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:29.912217 9150 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35797 } }
I20260807 20:08:29.912305 9150 leader_election.cc:302] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba993fccc3734c6bab5d17ffd5d6cdf9; no voters:
I20260807 20:08:29.912490 9150 leader_election.cc:288] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:29.912516 9154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:29.912554 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9075
I20260807 20:08:29.912660 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:29.912832 9150 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:29.912949 3134 external_mini_cluster.cc:1428] Setting key 8b3052280e4b7d315a5f22e162a02088
I20260807 20:08:29.913008 9154 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [term 1 LEADER]: Becoming Leader. State: Replica: ba993fccc3734c6bab5d17ffd5d6cdf9, State: Running, Role: LEADER
I20260807 20:08:29.913149 9154 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35797 } }
I20260807 20:08:29.913491 9154 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba993fccc3734c6bab5d17ffd5d6cdf9. Latest consensus state: current_term: 1 leader_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35797 } } }
I20260807 20:08:29.913579 9154 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:29.913780 9156 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba993fccc3734c6bab5d17ffd5d6cdf9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35797 } } }
I20260807 20:08:29.913875 9156 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:29.913955 9164 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:29.914533 9164 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:29.916527 9164 catalog_manager.cc:1347] Generated new cluster ID: 0526509039f74e8485a7ec5ea399ca8f
I20260807 20:08:29.916584 9164 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:29 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:29.924445 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:29.913631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58320 (local address 127.3.15.190:35797)
0807 20:08:29.914409 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:29.914420 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:29.914456 (+ 36us) server_negotiation.cc:407] Connection header received
0807 20:08:29.915136 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:29.915156 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:29.915221 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:29.915368 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:29.916180 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:29.917333 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:29.918161 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:29.918611 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:29.918731 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:29.921198 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:29.921202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:29.921203 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:29.921215 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:29.922882 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:29.923260 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:29.923268 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:29.923268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:29.923357 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:29.923565 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:29.923566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:29.923566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:29.923823 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:29.924010 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:29.924080 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 20:08:29.924228 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":727,"thread_start_us":80,"threads_started":1}
I20260807 20:08:29.926232 9164 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:29.926839 9164 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:29.934074 9164 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba993fccc3734c6bab5d17ffd5d6cdf9: Generated new TSK 0
I20260807 20:08:29.934263 9164 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:29.965360 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35797
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:30.070770 9178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:30.070968 9178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:30.070991 9178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:30.072140 9178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:30.072186 9178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:30.072240 9178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:30.073824 9178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:30.074076 9178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:30.074302 9178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:08:30.079442 9178 server_base.cc:1034] running on GCE node
W20260807 20:08:30.079414 9184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:30.079414 9183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:30.079434 9186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:30.079813 9178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:30.080005 9178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:30.081139 9178 hybrid_clock.cc:648] HybridClock initialized: now 1786133310081129 us; error 31 us; skew 500 ppm
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.083665 9178 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:30.084041 9178 webserver.cc:460] Webserver started at http://127.3.15.129:46075/ using document root <none> and password file <none>
I20260807 20:08:30.084256 9178 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:30.084297 9178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:30.084384 9178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:30.085474 9178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "859f6c4dcdf84d1c9a178c35fb00d084"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "97b35bae6ea261711fe22ea3c961b2ac"
server_key_iv: "bcfc4254302e2e80fd79d2eccd4b2e89"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.085786 9178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "859f6c4dcdf84d1c9a178c35fb00d084"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "97b35bae6ea261711fe22ea3c961b2ac"
server_key_iv: "bcfc4254302e2e80fd79d2eccd4b2e89"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.087037 9178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:08:30.087723 9193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.087884 9178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:30.087985 9178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "859f6c4dcdf84d1c9a178c35fb00d084"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "97b35bae6ea261711fe22ea3c961b2ac"
server_key_iv: "bcfc4254302e2e80fd79d2eccd4b2e89"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.088056 9178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:30.098284 9178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:30.100468 9178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:30.100626 9178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:30.100881 9178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:30.101193 9178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:30.101244 9178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.101287 9178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:30.101315 9178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.107097 9178 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:36923
I20260807 20:08:30.107174 9313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:36923 every 8 connection(s)
I20260807 20:08:30.107481 9178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:30.111073 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9178
I20260807 20:08:30.111202 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:30.111392 3134 external_mini_cluster.cc:1428] Setting key bd99718444884b5b35c80489e34b9886
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:30.117743 9316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.108176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35797 (local address 127.3.15.129:32981)
0807 20:08:30.108351 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:30.108387 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:30.109382 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:30.109615 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:30.109619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:30.109855 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:30.110039 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:30.110043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.110694 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.110694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:30.111654 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:30.111657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.112057 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.112059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:30.112184 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.112658 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:30.112666 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:30.114391 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:30.116310 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:30.116317 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:30.116319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:30.116535 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:30.116816 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:30.116817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:30.116818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:30.116948 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:30.117254 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:30.117259 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:30.117373 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 20:08:30.117446 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:08:30.117566 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260807 20:08:30.117789 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.108202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:32981 (local address 127.3.15.190:35797)
0807 20:08:30.108270 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.108273 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.108465 (+ 192us) server_negotiation.cc:407] Connection header received
0807 20:08:30.109491 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.109492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.109506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.109544 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.110127 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.110618 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.111744 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.111964 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.112039 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.114485 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:30.114486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:30.114487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:30.114498 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:30.116217 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.116620 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.116622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.116622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.116693 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.117019 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.117021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.117021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.117177 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:30.117249 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:30.117515 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 20:08:30.117641 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:08:30.118234 9314 heartbeater.cc:347] Connected to a master server at 127.3.15.190:35797
I20260807 20:08:30.118335 9314 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:30.118510 9314 heartbeater.cc:511] Master 127.3.15.190:35797 requested a full tablet report, sending...
I20260807 20:08:30.119112 9108 ts_manager.cc:194] Registered new tserver with Master: 859f6c4dcdf84d1c9a178c35fb00d084 (127.3.15.129:36923)
I20260807 20:08:30.120070 9108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:32981
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:30.139230 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35797
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:30.223383 9319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:30.223634 9319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:30.223665 9319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:30.225028 9319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:30.225073 9319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:30.225103 9319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:30.226801 9319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:30.227149 9319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:30.227416 9319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:30.232437 9325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:30.232412 9324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:30.232412 9327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:30.232808 9319 server_base.cc:1034] running on GCE node
I20260807 20:08:30.233057 9319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:30.233299 9319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:30.234460 9319 hybrid_clock.cc:648] HybridClock initialized: now 1786133310234447 us; error 31 us; skew 500 ppm
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.237273 9319 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:30.237730 9319 webserver.cc:460] Webserver started at http://127.3.15.130:37441/ using document root <none> and password file <none>
I20260807 20:08:30.237963 9319 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:30.238009 9319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:30.238173 9319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:30.239271 9319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "fad9ed69c30a4c9f8af0a3fd0a5024f9"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "1a71c596fa02bddf71d8c640b48cb685"
server_key_iv: "295e9de7cab9e0162da4cdf3099dd8a7"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.239634 9319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "fad9ed69c30a4c9f8af0a3fd0a5024f9"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "1a71c596fa02bddf71d8c640b48cb685"
server_key_iv: "295e9de7cab9e0162da4cdf3099dd8a7"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.241303 9319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:08:30.242071 9335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.242255 9319 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:30.242352 9319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "fad9ed69c30a4c9f8af0a3fd0a5024f9"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "1a71c596fa02bddf71d8c640b48cb685"
server_key_iv: "295e9de7cab9e0162da4cdf3099dd8a7"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.242422 9319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:30.256978 9319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:30.258759 9319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:30.258912 9319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:30.259155 9319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:30.259466 9319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:30.259503 9319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.259526 9319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:30.259572 9319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.265602 9319 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:44531
I20260807 20:08:30.265650 9456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:44531 every 8 connection(s)
I20260807 20:08:30.265980 9319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:30.274590 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9319
I20260807 20:08:30.274677 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:30.274926 3134 external_mini_cluster.cc:1428] Setting key 305befbcd02897f55bf2ec6a9ea69caf
I20260807 20:08:30.275648 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.266706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:55795 (local address 127.3.15.190:35797)
0807 20:08:30.266762 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.266764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.266974 (+ 210us) server_negotiation.cc:407] Connection header received
0807 20:08:30.268017 (+ 1043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.268018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.268031 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.268099 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.268734 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.269321 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.270346 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.270570 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.270647 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.272740 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:30.272741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:30.272742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:30.272749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:30.273966 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.274382 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.274383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.274384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.274471 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.274807 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.274809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.274809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.274972 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:30.275056 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:30.275431 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 20:08:30.275519 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:08:30.275785 9460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.266668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35797 (local address 127.3.15.130:55795)
0807 20:08:30.266848 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:30.266884 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:30.267893 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:30.268173 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:30.268177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:30.268444 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:30.268637 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:30.268641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.269429 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.269430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:30.270249 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:30.270252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.270653 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.270655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:30.270787 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.271255 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:30.271263 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:30.272642 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:30.274058 (+ 1416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:30.274065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:30.274067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:30.274293 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:30.274543 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:30.274544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:30.274544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:30.274673 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:30.275092 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:30.275099 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:30.275275 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 20:08:30.275389 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 20:08:30.275520 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260807 20:08:30.276271 9457 heartbeater.cc:347] Connected to a master server at 127.3.15.190:35797
I20260807 20:08:30.276379 9457 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:30.276572 9457 heartbeater.cc:511] Master 127.3.15.190:35797 requested a full tablet report, sending...
I20260807 20:08:30.276988 9108 ts_manager.cc:194] Registered new tserver with Master: fad9ed69c30a4c9f8af0a3fd0a5024f9 (127.3.15.130:44531)
I20260807 20:08:30.277365 9108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:55795
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:30.301750 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35797
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37191
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:30.384375 9463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:30.384570 9463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:30.384594 9463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:30.385799 9463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:30.385846 9463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:30.385905 9463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:30.387394 9463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37191
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:30.387629 9463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:30.387848 9463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:30.393018 9468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:30.392982 9473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:30.393198 9463 server_base.cc:1034] running on GCE node
W20260807 20:08:30.393296 9471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:30.393561 9463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:30.393816 9463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:30.394975 9463 hybrid_clock.cc:648] HybridClock initialized: now 1786133310394967 us; error 34 us; skew 500 ppm
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.397624 9463 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:30.398044 9463 webserver.cc:460] Webserver started at http://127.3.15.131:38427/ using document root <none> and password file <none>
I20260807 20:08:30.398295 9463 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:30.398362 9463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:30.398510 9463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:30.399564 9463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "77461ebdd32944dfa6fab656b0e30a27"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "b491fb82a49ee1b6b97b276696f83c0a"
server_key_iv: "1c0a015fe1a4529b542f7ff48143544f"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.399940 9463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "77461ebdd32944dfa6fab656b0e30a27"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "b491fb82a49ee1b6b97b276696f83c0a"
server_key_iv: "1c0a015fe1a4529b542f7ff48143544f"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.401408 9463 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260807 20:08:30.402194 9481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.402375 9463 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:30.402467 9463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "77461ebdd32944dfa6fab656b0e30a27"
format_stamp: "Formatted at 2026-08-07 20:08:30 on dist-test-slave-1x4g"
server_key: "b491fb82a49ee1b6b97b276696f83c0a"
server_key_iv: "1c0a015fe1a4529b542f7ff48143544f"
server_key_version: "encryptionkey@0"
I20260807 20:08:30.402544 9463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:30.413138 9463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:30.415743 9463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:30.415925 9463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:30.416177 9463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:30.416491 9463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:30.416543 9463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.416584 9463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:30.416647 9463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:30.422580 9463 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:44073
I20260807 20:08:30.422626 9606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:44073 every 8 connection(s)
I20260807 20:08:30.422955 9463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:30.427080 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9463
I20260807 20:08:30.427156 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:30.427345 3134 external_mini_cluster.cc:1428] Setting key 9ebbd1a88eb4cb9c93510d4cbcd21620
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:30.432397 9165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.423615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:54253 (local address 127.3.15.190:35797)
0807 20:08:30.423691 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.423694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.423886 (+ 192us) server_negotiation.cc:407] Connection header received
0807 20:08:30.425080 (+ 1194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.425082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.425093 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.425142 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.425730 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.426220 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.427286 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.427588 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.427654 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.429902 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:30.429904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:30.429904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:30.429910 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:30.431150 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.431514 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.431515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.431515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.431571 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.431835 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.431836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.431837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.431966 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:30.432029 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:30.432269 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 20:08:30.432333 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:30.432606 9609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.423594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:35797 (local address 127.3.15.131:54253)
0807 20:08:30.423771 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:30.423808 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:30.424985 (+ 1177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:30.425209 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:30.425213 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:30.425464 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:30.425664 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:30.425668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.426295 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.426296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:30.427120 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:30.427122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.427691 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.427692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:30.427845 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.428414 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:30.428422 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:30.429821 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:30.431230 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:30.431236 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:30.431238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:30.431450 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:30.431631 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:30.431632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:30.431632 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:30.431729 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:30.432026 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:30.432031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:30.432157 (+ 126us) client_negotiation.cc:769] Sending connection context
0807 20:08:30.432238 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 20:08:30.432358 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260807 20:08:30.433189 9607 heartbeater.cc:347] Connected to a master server at 127.3.15.190:35797
I20260807 20:08:30.433298 9607 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:30.433490 9607 heartbeater.cc:511] Master 127.3.15.190:35797 requested a full tablet report, sending...
I20260807 20:08:30.433876 9108 ts_manager.cc:194] Registered new tserver with Master: 77461ebdd32944dfa6fab656b0e30a27 (127.3.15.131:44073)
I20260807 20:08:30.434357 9108 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:54253
I20260807 20:08:30.439131 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/test-admin.keytab.
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.456823 3134 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 20:08:30.458662 9614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:30.457889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.3.15.129:36923)
0807 20:08:30.458094 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.458099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.458110 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:30.458138 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.458140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.458155 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.458227 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.458598 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41766: BlockingRecv error: recv got EOF from 127.0.0.1:41766 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
W20260807 20:08:30.459033 3134 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc' of type FILE
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.460187 3134 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.461180 3134 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:30.469235 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.461729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.3.15.129:36923)
0807 20:08:30.461794 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.461795 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.461810 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:08:30.461899 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.461900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.461913 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.461969 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.462362 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.463099 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.463848 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.464193 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.464270 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.466420 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:30.466430 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:30.466430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:30.466438 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:30.467862 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.468265 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.468269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.468270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.468334 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.468535 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.468535 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.468536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.468679 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:30.468855 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:30.469038 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 20:08:30.469117 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260807 20:08:30.471251 9614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:30.470677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41778 (local address 127.3.15.129:36923)
0807 20:08:30.470716 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.470718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.470767 (+ 49us) server_negotiation.cc:407] Connection header received
0807 20:08:30.470839 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.470840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.470850 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.470893 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.471201 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41778: BlockingRecv error: recv got EOF from 127.0.0.1:41778 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 20:08:30.471338 3134 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc' of type FILE
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.472304 3134 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.473191 3134 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:30.480043 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.473660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.3.15.129:36923)
0807 20:08:30.473719 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.473721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.473781 (+ 60us) server_negotiation.cc:407] Connection header received
0807 20:08:30.473847 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.473848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.473859 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.473914 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.474298 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.474772 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.475453 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.475658 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.475732 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.477759 (+ 2027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:30.477760 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:30.477761 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:30.477768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:30.478910 (+ 1142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.479279 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.479280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.479281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.479337 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.479540 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.479541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.479541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.479636 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:30.479730 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:30.479840 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 20:08:30.479917 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 20:08:30.481707 9614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:30.481053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41800 (local address 127.3.15.129:36923)
0807 20:08:30.481094 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.481096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.481151 (+ 55us) server_negotiation.cc:407] Connection header received
0807 20:08:30.481221 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.481222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.481231 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.481278 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.481659 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41800: BlockingRecv error: recv got EOF from 127.0.0.1:41800 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 20:08:30.481792 3134 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786133281907902-3134-0/krb5kdc/krb5cc' of type FILE
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.482807 3134 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:30.483680 3134 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:08:30.490620 9614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:30.484158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.3.15.129:36923)
0807 20:08:30.484219 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:30.484221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:30.484286 (+ 65us) server_negotiation.cc:407] Connection header received
0807 20:08:30.484375 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:30.484376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:30.484386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:30.484451 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:30.484841 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.485306 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.485979 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:30.486175 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:30.486257 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:30.488185 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:30.488186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:30.488187 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:30.488194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:30.489531 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.489867 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.489868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.489868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.489923 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:30.490113 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:30.490114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:30.490114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:30.490240 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:30.490337 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:30.490448 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 20:08:30.490521 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:08:30.491322 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9178
I20260807 20:08:30.497654 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9319
I20260807 20:08:30.503384 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9463
I20260807 20:08:30.510679 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9075
2026-08-07T20:08:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2028 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9627](info): set up 2 sockets
Aug 07 20:08:30 dist-test-slave-1x4g krb5kdc[9627](info): commencing operation
krb5kdc: starting...
W20260807 20:08:31.594349 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:31 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab.
Entry for principal oryx/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab.
I20260807 20:08:31.720402 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33055
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab
--principal=oryx/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:31.804102 9642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:31.804337 9642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:31.804373 9642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:31.805557 9642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:31.805620 9642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:31.805644 9642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:31.805666 9642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:31.805703 9642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:31.807209 9642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab
--principal=oryx/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9642
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:31.807489 9642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:31.807715 9642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:31.812659 9650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:31.812695 9642 server_base.cc:1034] running on GCE node
W20260807 20:08:31.812683 9647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:31.812647 9648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:31.813124 9642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:31.813439 9642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:31.814589 9642 hybrid_clock.cc:648] HybridClock initialized: now 1786133311814571 us; error 37 us; skew 500 ppm
Aug 07 20:08:31 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:31.820700 9642 init.cc:377] Logged in from keytab as oryx/127.3.15.190@KRBTEST.COM (short username oryx)
I20260807 20:08:31.821558 9642 webserver.cc:460] Webserver started at http://127.3.15.190:40683/ using document root <none> and password file <none>
I20260807 20:08:31.821843 9642 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:31.821899 9642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:31.822036 9642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:31.823107 9642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "23cafcb5ec1a429e9291a39af4c9e767"
format_stamp: "Formatted at 2026-08-07 20:08:31 on dist-test-slave-1x4g"
server_key: "fa4d256a71714aadbd2274fdb28012cb"
server_key_iv: "882aac27570903236bbd44ebce19f8f2"
server_key_version: "encryptionkey@0"
I20260807 20:08:31.823463 9642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "23cafcb5ec1a429e9291a39af4c9e767"
format_stamp: "Formatted at 2026-08-07 20:08:31 on dist-test-slave-1x4g"
server_key: "fa4d256a71714aadbd2274fdb28012cb"
server_key_iv: "882aac27570903236bbd44ebce19f8f2"
server_key_version: "encryptionkey@0"
I20260807 20:08:31.825177 9642 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 20:08:31.826195 9658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:31.826401 9642 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:31.826499 9642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "23cafcb5ec1a429e9291a39af4c9e767"
format_stamp: "Formatted at 2026-08-07 20:08:31 on dist-test-slave-1x4g"
server_key: "fa4d256a71714aadbd2274fdb28012cb"
server_key_iv: "882aac27570903236bbd44ebce19f8f2"
server_key_version: "encryptionkey@0"
I20260807 20:08:31.826574 9642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:31.848814 9642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:31.850881 9642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:31.851095 9642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:31.855214 9642 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:33055
I20260807 20:08:31.855269 9714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:33055 every 8 connection(s)
I20260807 20:08:31.855659 9642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:31.856731 9715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:31.859064 9715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Bootstrap starting.
I20260807 20:08:31.859612 9715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:31.859843 9715 log.cc:824] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:31.860502 9715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: No bootstrap required, opened a new log
I20260807 20:08:31.862075 9715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } }
I20260807 20:08:31.862234 9715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:31.862257 9715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23cafcb5ec1a429e9291a39af4c9e767, State: Initialized, Role: FOLLOWER
I20260807 20:08:31.862360 9715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } }
I20260807 20:08:31.862443 9715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:31.862476 9715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:31.862547 9715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:31.863360 9715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } }
I20260807 20:08:31.863477 9715 leader_election.cc:302] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23cafcb5ec1a429e9291a39af4c9e767; no voters:
I20260807 20:08:31.863651 9715 leader_election.cc:288] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:31.863695 9718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:31.863900 9715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:31.863919 9718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 LEADER]: Becoming Leader. State: Replica: 23cafcb5ec1a429e9291a39af4c9e767, State: Running, Role: LEADER
I20260807 20:08:31.864001 9718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } }
I20260807 20:08:31.864393 9719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } }
I20260807 20:08:31.864604 9719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:31.864530 9720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23cafcb5ec1a429e9291a39af4c9e767. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } }
I20260807 20:08:31.864820 9720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:31.865154 9727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:31.865708 9727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:31.865896 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9642
I20260807 20:08:31.866022 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:31.866219 3134 external_mini_cluster.cc:1428] Setting key d0670f405b5b608797085ed798aa38e1
I20260807 20:08:31.867300 9727 catalog_manager.cc:1347] Generated new cluster ID: 162a9861a0894381897d0107e34039b0
I20260807 20:08:31.867362 9727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:31 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:31.879783 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:31.866665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.3.15.190:33055)
0807 20:08:31.868897 (+ 2232us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:31.868907 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:31.868945 (+ 38us) server_negotiation.cc:407] Connection header received
0807 20:08:31.869069 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:31.869075 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:31.869130 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:31.869284 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:31.870022 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:31.871492 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:31.872522 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:31.873206 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:31.873325 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:31.876089 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:31.876093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:31.876094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:31.876106 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:31.878141 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:31.878556 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:31.878559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:31.878560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:31.878665 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:31.878873 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:31.878874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:31.878874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:31.879172 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:31.879293 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:31.879420 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 20:08:31.879555 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2179,"thread_start_us":79,"threads_started":1}
I20260807 20:08:31.885349 9727 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:31.886063 9727 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:31.901798 9727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Generated new TSK 0
I20260807 20:08:31.902019 9727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab.
Entry for principal oryx/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab.
I20260807 20:08:31.934044 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33055
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:32.018075 9743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:32.018261 9743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:32.018285 9743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:32.019387 9743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:32.019428 9743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:32.019452 9743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:32.020954 9743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:32.021198 9743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:32.021438 9743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.026371 9750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.026403 9752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.026568 9743 server_base.cc:1034] running on GCE node
W20260807 20:08:32.026626 9749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.026809 9743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.027017 9743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.028149 9743 hybrid_clock.cc:648] HybridClock initialized: now 1786133312028138 us; error 28 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.030655 9743 init.cc:377] Logged in from keytab as oryx/127.3.15.129@KRBTEST.COM (short username oryx)
I20260807 20:08:32.031170 9743 webserver.cc:460] Webserver started at http://127.3.15.129:44527/ using document root <none> and password file <none>
I20260807 20:08:32.031409 9743 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.031453 9743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.031610 9743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:32.032649 9743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "eac1c7997cf5407b91acb8d2c8d15500"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "445d86a772a302b187866a9bcd1d7752"
server_key_iv: "6d7757a6ed68aff007117b0ad7301588"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.033092 9743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "eac1c7997cf5407b91acb8d2c8d15500"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "445d86a772a302b187866a9bcd1d7752"
server_key_iv: "6d7757a6ed68aff007117b0ad7301588"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.034394 9743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:32.035101 9760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.035269 9743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:32.035370 9743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "eac1c7997cf5407b91acb8d2c8d15500"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "445d86a772a302b187866a9bcd1d7752"
server_key_iv: "6d7757a6ed68aff007117b0ad7301588"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.035437 9743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.048954 9743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.050810 9743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:32.050966 9743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.051191 9743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:32.051507 9743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:32.051565 9743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.051600 9743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:32.051625 9743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.057600 9743 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40975
I20260807 20:08:32.057643 9881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40975 every 8 connection(s)
I20260807 20:08:32.057973 9743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:32.059940 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9743
I20260807 20:08:32.060047 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:32.060271 3134 external_mini_cluster.cc:1428] Setting key 6e77ac8d5889289badac40b1e7375d78
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:32.069199 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.058731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:33335 (local address 127.3.15.190:33055)
0807 20:08:32.058795 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.058797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.058989 (+ 192us) server_negotiation.cc:407] Connection header received
0807 20:08:32.059489 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.059490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.059504 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.059543 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.060640 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.061290 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.062884 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.063211 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.063286 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.066088 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.066089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.066090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.066096 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.067513 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.068045 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.068046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.068046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.068110 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.068455 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.068456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.068457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.068575 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.068635 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.069034 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.069130 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:08:32.069450 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.058706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:33335)
0807 20:08:32.058863 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.058905 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.059390 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.059608 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.059611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.060275 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.060527 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.060534 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.061384 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.061385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.062757 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.062761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.063307 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.063308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.063419 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.063979 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.063987 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.065984 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.067605 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.067614 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.067617 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.067947 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.068218 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.068219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.068219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.068377 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.068647 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.068656 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.068864 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.068990 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.069176 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260807 20:08:32.070060 9882 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33055
I20260807 20:08:32.070170 9882 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.3.15.130@KRBTEST.COM; defaulting to no policy
I20260807 20:08:32.070384 9882 heartbeater.cc:511] Master 127.3.15.190:33055 requested a full tablet report, sending...
I20260807 20:08:32.070947 9675 ts_manager.cc:194] Registered new tserver with Master: eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975)
I20260807 20:08:32.071600 9675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:33335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab.
Entry for principal oryx/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab.
I20260807 20:08:32.088370 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33055
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:32.173478 9887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:32.173727 9887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:32.173753 9887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:32.174834 9887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:32.174868 9887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:32.174893 9887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:32.176283 9887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:32.176463 9887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:32.176646 9887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.181512 9895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.181525 9893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.181516 9892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.181828 9887 server_base.cc:1034] running on GCE node
I20260807 20:08:32.182021 9887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.182228 9887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.183365 9887 hybrid_clock.cc:648] HybridClock initialized: now 1786133312183353 us; error 29 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.186107 9887 init.cc:377] Logged in from keytab as oryx/127.3.15.130@KRBTEST.COM (short username oryx)
I20260807 20:08:32.186558 9887 webserver.cc:460] Webserver started at http://127.3.15.130:42603/ using document root <none> and password file <none>
I20260807 20:08:32.186782 9887 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.186847 9887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.186995 9887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:32.188022 9887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "e286b497d371491ba4c552c0f9eb285b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "493d08a5f97829be3bc840e47eac7763"
server_key_iv: "4fcb5ae99510c518d282b77f7b77318f"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.188392 9887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "e286b497d371491ba4c552c0f9eb285b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "493d08a5f97829be3bc840e47eac7763"
server_key_iv: "4fcb5ae99510c518d282b77f7b77318f"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.189785 9887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:32.190508 9902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.190665 9887 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:32.190764 9887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "e286b497d371491ba4c552c0f9eb285b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "493d08a5f97829be3bc840e47eac7763"
server_key_iv: "4fcb5ae99510c518d282b77f7b77318f"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.190857 9887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.206718 9887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.209038 9887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:32.209206 9887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.209436 9887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:32.209738 9887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:32.209787 9887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.209830 9887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:32.209858 9887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.215564 9887 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:45133
I20260807 20:08:32.215615 10030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:45133 every 8 connection(s)
I20260807 20:08:32.215981 9887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:32.224489 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 9887
I20260807 20:08:32.224581 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:32.224835 3134 external_mini_cluster.cc:1428] Setting key 6317228fd352039411e26ace54865d49
I20260807 20:08:32.225492 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.216701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:49971 (local address 127.3.15.190:33055)
0807 20:08:32.216769 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.216771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.216986 (+ 215us) server_negotiation.cc:407] Connection header received
0807 20:08:32.217473 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.217474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.217485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.217539 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.218527 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.219066 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.220154 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.220365 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.220432 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.222576 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.222577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.222578 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.222592 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.223891 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.224285 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.224286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.224287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.224364 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.224702 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.224708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.224708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.224875 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.225002 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.225305 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.225386 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:08:32.225651 10034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.216668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:49971)
0807 20:08:32.216848 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.216898 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.217379 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.217605 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.217609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.218241 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.218436 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.218442 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.219176 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.219177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.220040 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.220043 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.220452 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.220452 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.220563 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.221061 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.221070 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.222469 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.223975 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.223982 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.223985 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.224211 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.224452 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.224453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.224453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.224620 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.224982 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.224990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.225132 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.225256 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.225425 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":107,"threads_started":1}
I20260807 20:08:32.226156 10031 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33055
I20260807 20:08:32.226262 10031 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:32.226452 10031 heartbeater.cc:511] Master 127.3.15.190:33055 requested a full tablet report, sending...
I20260807 20:08:32.226878 9675 ts_manager.cc:194] Registered new tserver with Master: e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133)
I20260807 20:08:32.227389 9675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:49971
WARNING: no policy specified for oryx/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab.
Entry for principal oryx/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab.
I20260807 20:08:32.251866 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33055
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:32.336562 10037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:32.336856 10037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:32.336890 10037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:32.338020 10037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:32.338055 10037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:32.338083 10037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:32.339550 10037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:33055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:32.339741 10037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:32.339924 10037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.344859 10043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.345029 10037 server_base.cc:1034] running on GCE node
W20260807 20:08:32.344890 10046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.345146 10044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.345350 10037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.345592 10037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.346733 10037 hybrid_clock.cc:648] HybridClock initialized: now 1786133312346722 us; error 24 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.349380 10037 init.cc:377] Logged in from keytab as oryx/127.3.15.131@KRBTEST.COM (short username oryx)
I20260807 20:08:32.349849 10037 webserver.cc:460] Webserver started at http://127.3.15.131:33279/ using document root <none> and password file <none>
I20260807 20:08:32.350157 10037 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.350227 10037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.350371 10037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:32.351455 10037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "52da89c27ef146a19f5355741440587b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "bf2ce823608789162f5f5ce3cbe5c15a"
server_key_iv: "4eb7ce9633ae6a9e4d20bb3e32e54314"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.351821 10037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "52da89c27ef146a19f5355741440587b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "bf2ce823608789162f5f5ce3cbe5c15a"
server_key_iv: "4eb7ce9633ae6a9e4d20bb3e32e54314"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.353315 10037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:32.354090 10053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.354266 10037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:32.354377 10037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "52da89c27ef146a19f5355741440587b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "bf2ce823608789162f5f5ce3cbe5c15a"
server_key_iv: "4eb7ce9633ae6a9e4d20bb3e32e54314"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.354452 10037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.386404 10037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.388950 10037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:32.389119 10037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.389364 10037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:32.389672 10037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:32.389722 10037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.389765 10037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:32.389796 10037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.395704 10037 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:42541
I20260807 20:08:32.395800 10176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:42541 every 8 connection(s)
I20260807 20:08:32.396173 10037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:32.397552 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 10037
I20260807 20:08:32.397634 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:32.397814 3134 external_mini_cluster.cc:1428] Setting key 9506c2094aada33c057576c9e1cfeb70
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:32.405748 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.397027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:49797 (local address 127.3.15.190:33055)
0807 20:08:32.397078 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.397080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.397297 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:08:32.397872 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.397873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.397886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.397951 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.399137 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.399648 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.400666 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.400915 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.400997 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.403156 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.403157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.403158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.403165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.404380 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.404804 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.404805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.404805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.404868 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.405119 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.405120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.405121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.405235 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.405306 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.405576 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.405656 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:32.405946 10179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.396993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:49797)
0807 20:08:32.397169 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.397212 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.397761 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.398028 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.398032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.398836 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.399033 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.399039 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.399741 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.399742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.400564 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.400566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.401009 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.401010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.401148 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.401562 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.401570 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.403060 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.404468 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.404475 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.404477 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.404701 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.404934 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.404935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.404935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.405041 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.405299 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.405305 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.405441 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.405540 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.405708 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260807 20:08:32.406418 10177 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33055
I20260807 20:08:32.406507 10177 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:32.406651 10177 heartbeater.cc:511] Master 127.3.15.190:33055 requested a full tablet report, sending...
I20260807 20:08:32.407019 9675 ts_manager.cc:194] Registered new tserver with Master: 52da89c27ef146a19f5355741440587b (127.3.15.131:42541)
I20260807 20:08:32.407338 9675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:49797
I20260807 20:08:32.409571 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.15.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.189.keytab.
Entry for principal oryx/127.3.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.189.keytab.
I20260807 20:08:32.436059 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:36985
--webserver_interface=127.3.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.189.keytab
--principal=oryx/127.3.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33055,127.3.15.189:36985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:32.520540 10182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:32.520897 10182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:32.520955 10182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:32.522140 10182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:32.522207 10182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:32.522233 10182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:32.522269 10182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:32.522302 10182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:32.523792 10182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33055,127.3.15.189:36985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.189.keytab
--principal=oryx/127.3.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:36985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
W20260807 20:08:32.523953 10182 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.15.190:33055,127.3.15.189:36985'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 20:08:32.524108 10182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:32.524334 10182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.529743 10189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.529918 10182 server_base.cc:1034] running on GCE node
W20260807 20:08:32.529677 10191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.529716 10188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.530345 10182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.530685 10182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.531874 10182 hybrid_clock.cc:648] HybridClock initialized: now 1786133312531870 us; error 54 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.534678 10182 init.cc:377] Logged in from keytab as oryx/127.3.15.189@KRBTEST.COM (short username oryx)
I20260807 20:08:32.535162 10182 webserver.cc:460] Webserver started at http://127.3.15.189:41691/ using document root <none> and password file <none>
I20260807 20:08:32.535416 10182 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.535481 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.535620 10182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:32.536705 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/instance:
uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "a1584d17bd025918b25e231f0a471061"
server_key_iv: "19a75b80ecfd079ada5f95327a957173"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.537130 10182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal/instance:
uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "a1584d17bd025918b25e231f0a471061"
server_key_iv: "19a75b80ecfd079ada5f95327a957173"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.538550 10182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:32.539268 10198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.539429 10182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:32.539535 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "a1584d17bd025918b25e231f0a471061"
server_key_iv: "19a75b80ecfd079ada5f95327a957173"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.539603 10182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.551589 10182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.553421 10182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:32.553575 10182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.557739 10182 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.189:36985
I20260807 20:08:32.557775 10253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.189:36985 every 8 connection(s)
I20260807 20:08:32.558128 10182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
I20260807 20:08:32.559204 10254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:32.560751 10254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:08:32.561326 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 10182
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.189@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
I20260807 20:08:32.570399 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.561680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53742 (local address 127.3.15.190:33055)
0807 20:08:32.561731 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.561733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.561877 (+ 144us) server_negotiation.cc:407] Connection header received
0807 20:08:32.562460 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.562461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.562476 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.562537 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.563075 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.563604 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.564585 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.564833 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.564919 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.567197 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.567199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.567200 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.567208 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.568884 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.569305 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.569306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.569307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.569375 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.569630 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.569631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.569631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.569799 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.569855 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.570197 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.570289 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:08:32.570611 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.561511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:53742)
0807 20:08:32.561730 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.561781 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.562350 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.562557 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.562560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.562826 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.562984 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.562989 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.563707 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.563708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.564487 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.564489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.564916 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.564917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.565056 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.565568 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.565577 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.567089 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.568962 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.568968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.568969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.569223 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.569439 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.569439 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.569439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.569540 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.569866 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.569872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.570035 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.570175 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.570353 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":20,"thread_start_us":118,"threads_started":1}
I20260807 20:08:32.571132 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.561855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.3.15.189:36985)
0807 20:08:32.561983 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.561988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.562173 (+ 185us) server_negotiation.cc:407] Connection header received
0807 20:08:32.562350 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.562356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.562376 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.562471 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.563076 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.563899 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.564765 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.565204 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.565289 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.567666 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.567674 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.567674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.567681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.569550 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.569982 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.569985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.569985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.570037 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.570253 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.570254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.570254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.570411 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.570553 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.570984 (+ 431us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.571065 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260807 20:08:32.571501 10254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.189@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
I20260807 20:08:32.577590 10256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.571753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44248)
0807 20:08:32.571830 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.571841 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.571898 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.572017 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.572018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.572143 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.572171 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.572172 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.572833 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.572834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.573305 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.573306 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.573688 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.573689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.573742 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.574074 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.574079 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.575308 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.576703 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.576704 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.576705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.576912 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.577122 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.577122 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.577123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.577177 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.577352 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.577353 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.577388 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.577437 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.577513 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 20:08:32.577687 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.571787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44248 (local address 127.3.15.189:36985)
0807 20:08:32.571833 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.571835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.571891 (+ 56us) server_negotiation.cc:407] Connection header received
0807 20:08:32.571954 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.571955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.571964 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.572006 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.572235 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.572702 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.573395 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.573610 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.573688 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.575381 (+ 1693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.575382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.575382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.575388 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.576629 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.577003 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.577004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.577005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.577059 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.577215 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.577215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.577216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.577308 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.577351 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.577454 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.577556 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":25}
I20260807 20:08:32.579130 10254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Bootstrap starting.
I20260807 20:08:32.579983 10254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:32.580271 10254 log.cc:824] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:32.581092 10254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: No bootstrap required, opened a new log
I20260807 20:08:32.582490 10254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } }
I20260807 20:08:32.582633 10254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:32.582690 10254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68ad022a655a483f9f322e8bb7dfb0c5, State: Initialized, Role: FOLLOWER
I20260807 20:08:32.582858 10254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } }
I20260807 20:08:32.583182 10254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:32.583150 10260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } } }
I20260807 20:08:32.583268 10260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:08:32.584867 10271 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:08:32.584928 10271 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:08:32.587106 10182 master_runner.cc:428] Detected that this master 68ad022a655a483f9f322e8bb7dfb0c5 is joining an existing cluster
I20260807 20:08:32.587173 10182 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.15.189:36985
I20260807 20:08:32.595005 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.588183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44262)
0807 20:08:32.588372 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.588398 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.588462 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.588620 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.588621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.588871 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.588974 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.588977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.589920 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.589921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.590825 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.590828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.591303 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.591304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.591352 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.591737 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.591742 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.592108 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.593971 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.593972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.593972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.594140 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.594357 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.594358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.594420 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.594675 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.594676 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.594720 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.594787 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.594897 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260807 20:08:32.595357 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.588358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44262 (local address 127.3.15.189:36985)
0807 20:08:32.588417 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.588419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.588445 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:08:32.588519 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.588520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.588530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.588591 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.589058 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.589836 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.590913 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.591198 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.591266 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.592180 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.592181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.592182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.592187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.593879 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594221 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.594222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.594222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.594294 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594481 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.594481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.594482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.594618 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.594685 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.594814 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.595269 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":371,"server-negotiator.queue_time_us":38}
I20260807 20:08:32.595763 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.588281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.3.15.190:33055)
0807 20:08:32.588328 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.588330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.588610 (+ 280us) server_negotiation.cc:407] Connection header received
0807 20:08:32.588685 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.588686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.588697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.588745 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.589150 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.589905 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.590648 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.590861 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.590941 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.592129 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.592131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.592131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.592140 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.594186 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594639 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.594640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.594640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.594703 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594901 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.594902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.594902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.595022 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.595195 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.595576 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.595664 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:08:32.596040 10278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.588188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:53756)
0807 20:08:32.588555 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.588563 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.588632 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.588759 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.588760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.588976 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.589051 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.589053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.589996 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.589997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.590564 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.590565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.591013 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.591014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.591074 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.591579 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.591584 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.592028 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.594308 (+ 2280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.594309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.594555 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.594762 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.594763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.594763 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.594821 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.595363 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.595364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.595427 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.595843 (+ 416us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.595941 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":90,"threads_started":1}
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.190@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
I20260807 20:08:32.606769 10257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.598439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44268 (local address 127.3.15.189:36985)
0807 20:08:32.598480 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.598482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.598595 (+ 113us) server_negotiation.cc:407] Connection header received
0807 20:08:32.598704 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.598705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.598712 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.598752 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.599275 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.600035 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.601357 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.601548 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.601610 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.603988 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.603990 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.603990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.603997 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.605449 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.605924 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.605924 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.605925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.605976 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.606225 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.606226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.606226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.606346 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.606458 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.606641 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.606708 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:32.606871 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.598352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44268)
0807 20:08:32.598500 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.598545 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.598630 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.598767 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.598770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.599115 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.599176 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.599178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.600142 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.600143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.601187 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.601190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.601632 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.601633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.601721 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.602210 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.602215 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.603855 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.605527 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.605535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.605538 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.605825 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.606037 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.606038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.606038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.606139 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.606454 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.606462 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.606519 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.606620 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.606748 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":119,"threads_started":1}
I20260807 20:08:32.607659 9675 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.15.189:36985: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
member_type: NON_VOTER
last_known_addr {
host: "127.3.15.189"
port: 36985
}
attrs {
promote: true
}
}
dest_uuid: "23cafcb5ec1a429e9291a39af4c9e767"
cas_config_opid_index: -1
I20260807 20:08:32.608042 9675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } }
I20260807 20:08:32.608548 10281 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 68ad022a655a483f9f322e8bb7dfb0c5 (127.3.15.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } }
I20260807 20:08:32.608896 10227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:32.609757 10281 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.15.189:36985
I20260807 20:08:32.609760 10227 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Refusing update from remote peer 23cafcb5ec1a429e9291a39af4c9e767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 20:08:32.610066 10182 master.cc:537] Master@127.3.15.189:36985 shutting down...
I20260807 20:08:32.610083 10281 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:32.610579 10280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } }
I20260807 20:08:32.610742 10280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
W20260807 20:08:32.610773 9661 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 20:08:32.611305 9661 consensus_peers.cc:487] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 -> Peer 68ad022a655a483f9f322e8bb7dfb0c5 (127.3.15.189:36985): Couldn't send request to peer 68ad022a655a483f9f322e8bb7dfb0c5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 20:08:32.613128 10182 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:32.613224 10182 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:32.613281 10182 tablet_replica.cc:330] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: stopping tablet replica
I20260807 20:08:32.626289 10182 master.cc:552] Master@127.3.15.189:36985 shutdown complete.
I20260807 20:08:32.626345 10182 master_runner.cc:305] Clearing existing system tablet
I20260807 20:08:32.626490 10182 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.626549 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.627213 10182 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260807 20:08:32.627674 10286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.627785 10182 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.627843 10182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.627866 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "a1584d17bd025918b25e231f0a471061"
server_key_iv: "19a75b80ecfd079ada5f95327a957173"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.628614 10182 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:32.630139 10182 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 20:08:32.630199 10182 log.cc:1196] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 20:08:32.630405 10182 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Deleting consensus metadata
I20260807 20:08:32.630663 10182 master_runner.cc:315] Copying system tablet from 127.3.15.190:33055
I20260807 20:08:32.631268 10182 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: tablet copy: Beginning tablet copy session from remote peer at address 127.3.15.190:33055
I20260807 20:08:32.638454 10292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.631643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:53764)
0807 20:08:32.631764 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.631775 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.631865 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.632045 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.632046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.632225 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.632282 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.632284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.633339 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.633341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.633935 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.633938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.634515 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.634516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.634573 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.635117 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.635122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.635420 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.637353 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.637354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.637355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.637551 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.637804 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.637804 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.637805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.637860 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.638175 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.638176 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.638226 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.638285 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.638368 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":74,"threads_started":1}
I20260807 20:08:32.638638 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.631682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.3.15.190:33055)
0807 20:08:32.631731 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.631733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.631847 (+ 114us) server_negotiation.cc:407] Connection header received
0807 20:08:32.631950 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.631952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.631966 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.632025 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.632350 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.633241 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.634024 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.634421 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.634501 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.635524 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.635526 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.635526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.635534 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.637274 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.637631 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.637632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.637633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.637733 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.637922 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.637923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.637923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.638088 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.638176 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.638331 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.638426 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:32.638995 9697 tablet_copy_service.cc:140] P 23cafcb5ec1a429e9291a39af4c9e767: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 68ad022a655a483f9f322e8bb7dfb0c5 ({username='oryx', principal='oryx/127.3.15.189@KRBTEST.COM'} at 127.0.0.1:53764)
I20260807 20:08:32.639102 9697 tablet_copy_service.cc:161] P 23cafcb5ec1a429e9291a39af4c9e767: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 68ad022a655a483f9f322e8bb7dfb0c5 at {username='oryx', principal='oryx/127.3.15.189@KRBTEST.COM'} at 127.0.0.1:53764: session id = 68ad022a655a483f9f322e8bb7dfb0c5-00000000000000000000000000000000
I20260807 20:08:32.640102 9697 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 20:08:32.640765 10182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 20:08:32.641942 10182 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: tablet copy: Starting download of 0 data blocks...
I20260807 20:08:32.642127 10182 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: tablet copy: Starting download of 1 WAL segments...
I20260807 20:08:32.643213 10182 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 20:08:32.644042 9697 tablet_copy_service.cc:342] P 23cafcb5ec1a429e9291a39af4c9e767: Request end of tablet copy session 68ad022a655a483f9f322e8bb7dfb0c5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.15.189@KRBTEST.COM'} at 127.0.0.1:53764
I20260807 20:08:32.644110 9697 tablet_copy_service.cc:433] P 23cafcb5ec1a429e9291a39af4c9e767: ending tablet copy session 68ad022a655a483f9f322e8bb7dfb0c5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 68ad022a655a483f9f322e8bb7dfb0c5
I20260807 20:08:32.646791 10182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.647923 10293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.647980 10294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.648150 10296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.648233 10182 server_base.cc:1034] running on GCE node
I20260807 20:08:32.648367 10182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.648525 10182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.649650 10182 hybrid_clock.cc:648] HybridClock initialized: now 1786133312649645 us; error 31 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.651028 10182 init.cc:377] Logged in from keytab as oryx/127.3.15.189@KRBTEST.COM (short username oryx)
I20260807 20:08:32.651301 10182 webserver.cc:460] Webserver started at http://127.3.15.189:43957/ using document root <none> and password file <none>
I20260807 20:08:32.651414 10182 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.651443 10182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.652070 10182 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.652585 10303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.652704 10182 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.652751 10182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "a1584d17bd025918b25e231f0a471061"
server_key_iv: "19a75b80ecfd079ada5f95327a957173"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.652844 10182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.662760 10182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.664594 10182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.668227 10182 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.189:36985
I20260807 20:08:32.668519 10360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.189:36985 every 8 connection(s)
I20260807 20:08:32.668642 10182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
I20260807 20:08:32.669509 10361 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:08:32.669967 10361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Bootstrap starting.
I20260807 20:08:32.672505 10361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:08:32.672976 10361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: Bootstrap complete.
I20260807 20:08:32.673208 10361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } }
I20260807 20:08:32.673306 10361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 68ad022a655a483f9f322e8bb7dfb0c5, State: Initialized, Role: LEARNER
I20260807 20:08:32.673352 10361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } }
I20260807 20:08:32.673548 10361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:32.673547 10365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } }
I20260807 20:08:32.673810 10365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: LEARNER
I20260807 20:08:32.675088 10378 catalog_manager.cc:1259] Loaded cluster ID: 162a9861a0894381897d0107e34039b0
I20260807 20:08:32.675292 10378 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: loading cluster ID for follower catalog manager: success
I20260807 20:08:32.676455 10378 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: acquiring CA information for follower catalog manager: success
I20260807 20:08:32.677227 10378 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:08:32.680256 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.672799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44298 (local address 127.3.15.189:36985)
0807 20:08:32.672954 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.672956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.672965 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:32.672990 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.672992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.673003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.673797 (+ 794us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.673835 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.674889 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.675937 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.676260 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.676642 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.677697 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.677699 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.677699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.677706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.679084 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.679426 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.679427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.679427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.679513 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.679708 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.679709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.679709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.679820 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.679924 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.680063 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.680155 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for oryx/127.3.15.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.188.keytab.
Entry for principal oryx/127.3.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.188.keytab.
I20260807 20:08:32.712388 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:44985
--webserver_interface=127.3.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.188.keytab
--principal=oryx/127.3.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:32.817099 10382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:32.817308 10382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:32.817333 10382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:32.818483 10382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:32.818527 10382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:32.818542 10382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:32.818554 10382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:32.818567 10382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:32.820103 10382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.188.keytab
--principal=oryx/127.3.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:44985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:32.820361 10382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:32.820603 10382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.825629 10391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.825603 10388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.825661 10389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.825769 10382 server_base.cc:1034] running on GCE node
I20260807 20:08:32.826169 10382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.826423 10382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.827636 10382 hybrid_clock.cc:648] HybridClock initialized: now 1786133312827615 us; error 36 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.830542 10382 init.cc:377] Logged in from keytab as oryx/127.3.15.188@KRBTEST.COM (short username oryx)
I20260807 20:08:32.831179 10382 webserver.cc:460] Webserver started at http://127.3.15.188:43369/ using document root <none> and password file <none>
I20260807 20:08:32.831463 10382 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.831539 10382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.831719 10382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:32.833227 10382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/instance:
uuid: "4df1e015721641e3902f224ee4cd99f5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "3ff34f653bc9cfea00b8549865bf7a1c"
server_key_iv: "adef34277dbd69503991ba7b30fb66b8"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.833696 10382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal/instance:
uuid: "4df1e015721641e3902f224ee4cd99f5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "3ff34f653bc9cfea00b8549865bf7a1c"
server_key_iv: "adef34277dbd69503991ba7b30fb66b8"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.835436 10382 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:32.836263 10398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.836483 10382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:32.836588 10382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
uuid: "4df1e015721641e3902f224ee4cd99f5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "3ff34f653bc9cfea00b8549865bf7a1c"
server_key_iv: "adef34277dbd69503991ba7b30fb66b8"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.836695 10382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.853243 10382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.855391 10382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:32.855553 10382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.860831 10382 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.188:44985
I20260807 20:08:32.860948 10452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.188:44985 every 8 connection(s)
I20260807 20:08:32.861268 10382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
I20260807 20:08:32.862277 10453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:32.863868 10453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.188@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:32.870033 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 10382
I20260807 20:08:32.873258 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.864710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53772 (local address 127.3.15.190:33055)
0807 20:08:32.864765 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.864767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.864969 (+ 202us) server_negotiation.cc:407] Connection header received
0807 20:08:32.865420 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.865421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.865433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.865508 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.866144 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.866663 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.867680 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.867865 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.867964 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.870269 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.870271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.870272 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.870281 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.871843 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.872287 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.872288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.872288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.872340 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.872586 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.872586 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.872587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.872686 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.872763 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.873078 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.873171 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:08:32.873481 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.864567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:53772)
0807 20:08:32.864741 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.864813 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.865341 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.865582 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.865585 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.865838 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.866052 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.866057 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.866752 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.866753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.867594 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.867596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.867946 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.867947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.868045 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.868559 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.868568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.870169 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.871921 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.871926 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.871928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.872214 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.872398 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.872399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.872399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.872527 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.872769 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.872803 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.872938 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.873038 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.873193 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260807 20:08:32.874197 10453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.15.188@KRBTEST.COM
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.188@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
I20260807 20:08:32.880101 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.870549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48320 (local address 127.3.15.188:44985)
0807 20:08:32.870704 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.870709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.870731 (+ 22us) server_negotiation.cc:407] Connection header received
0807 20:08:32.870861 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.870864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.870875 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.871077 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.871662 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.872625 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.873468 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.874377 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.874449 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.876973 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.876981 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.876981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.876989 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.878883 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.879284 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.879287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.879287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.879343 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.879516 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.879517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.879517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.879666 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.879831 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.879909 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.879984 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260807 20:08:32.881695 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.874464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44302)
0807 20:08:32.874762 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.874771 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.874811 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.874942 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.874942 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.875107 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.875165 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.875166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.875857 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.875858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.876415 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.876417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.876856 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.876857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.876934 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.877318 (+ 384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.877323 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.878837 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.880963 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.880964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.880964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.881129 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.881281 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.881281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.881281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.881325 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.881504 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.881505 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.881535 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.881579 (+ 44us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.881640 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283}
I20260807 20:08:32.881721 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.874541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.3.15.189:36985)
0807 20:08:32.874596 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.874598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.874816 (+ 218us) server_negotiation.cc:407] Connection header received
0807 20:08:32.874853 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.874854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.874867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.874939 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.875253 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.875753 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.876505 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.876742 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.876855 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.878953 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.878955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.878956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.878963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.880899 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.881189 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.881190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.881190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.881242 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.881369 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.881369 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.881369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.881467 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.881514 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.881596 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.881660 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:08:32.882262 10453 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.188@KRBTEST.COM for oryx/127.3.15.188@KRBTEST.COM
I20260807 20:08:32.887805 10455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.882496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.0.0.1:48328)
0807 20:08:32.882525 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.882534 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.882569 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.882670 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.882671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.882774 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.882803 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.882804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.883355 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.883355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.883800 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.883801 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.884098 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.884099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.884135 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.884432 (+ 297us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.884435 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.885500 (+ 1065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.887035 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.887035 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.887036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.887188 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.887356 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.887357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.887357 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.887407 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.887570 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.887571 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.887601 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.887650 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.887718 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260807 20:08:32.887849 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.882528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48328 (local address 127.3.15.188:44985)
0807 20:08:32.882574 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.882575 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.882586 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:32.882620 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.882620 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.882630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.882666 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.882852 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.883292 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.883859 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.884041 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.884087 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.885581 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.885582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.885582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.885588 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.886906 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.887249 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.887250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.887250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.887300 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.887445 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.887445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.887445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.887526 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.887568 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.887666 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.887761 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":29}
I20260807 20:08:32.889279 10453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Bootstrap starting.
I20260807 20:08:32.890246 10453 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:32.890534 10453 log.cc:824] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:32.891279 10453 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: No bootstrap required, opened a new log
I20260807 20:08:32.892926 10453 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } }
I20260807 20:08:32.893064 10453 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:32.893114 10453 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4df1e015721641e3902f224ee4cd99f5, State: Initialized, Role: FOLLOWER
I20260807 20:08:32.893241 10453 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } }
I20260807 20:08:32.893548 10453 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:32.893515 10458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } } }
I20260807 20:08:32.893632 10458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:08:32.895368 10472 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:08:32.895423 10472 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:08:32.897774 10382 master_runner.cc:428] Detected that this master 4df1e015721641e3902f224ee4cd99f5 is joining an existing cluster
I20260807 20:08:32.897817 10382 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.15.188:44985
I20260807 20:08:32.905987 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.899200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44314 (local address 127.3.15.189:36985)
0807 20:08:32.899280 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.899282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.899823 (+ 541us) server_negotiation.cc:407] Connection header received
0807 20:08:32.899894 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.899895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.899907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.899973 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.900849 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.901316 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.902095 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.902297 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.902365 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.903328 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.903329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.903330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.903337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.904732 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.905167 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.905172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.905172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.905251 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.905465 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.905466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.905466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.905578 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.905646 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.905822 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.905895 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 20:08:32.905998 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.899079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44314)
0807 20:08:32.899618 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.899636 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.899830 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.899992 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.899993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.900321 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.900424 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.900425 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.901404 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.901405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.902013 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.902015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.902388 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.902389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.902452 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.902887 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.902892 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.903223 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.904852 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.904853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.904853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.905045 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.905326 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.905327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.905327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.905394 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.905649 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.905650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.905693 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.905783 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.905888 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":39,"thread_start_us":188,"threads_started":1}
I20260807 20:08:32.912271 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.899086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53780 (local address 127.3.15.190:33055)
0807 20:08:32.899137 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.899139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.905140 (+ 6001us) server_negotiation.cc:407] Connection header received
0807 20:08:32.905221 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.905222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.905236 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.905297 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.906604 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.907383 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.908511 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.908705 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.908764 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.909792 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.909793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.909793 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.909800 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.911194 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.911574 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.911574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.911575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.911632 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.911822 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.911822 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.911822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.911928 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.911977 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.912146 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.912206 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:32.912349 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.899296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:53780)
0807 20:08:32.904877 (+ 5581us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.904889 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.905150 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.905307 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.905308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.905983 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.906491 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.906494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.907484 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.907485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.908420 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.908423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.908795 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.908796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.908865 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.909285 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.909290 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.909693 (+ 403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.911273 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.911274 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.911274 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.911502 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.911693 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.911694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.911694 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.911762 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.911984 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.911985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.912052 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.912130 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.912232 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5560,"thread_start_us":246,"threads_started":1}
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133311, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.190@KRBTEST.COM for oryx/127.3.15.188@KRBTEST.COM
I20260807 20:08:32.924443 10456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.899197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48330 (local address 127.3.15.188:44985)
0807 20:08:32.899248 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.899250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.899772 (+ 522us) server_negotiation.cc:407] Connection header received
0807 20:08:32.899873 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.899874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.899885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.899911 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.900505 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.901274 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.902369 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.902638 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.902685 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.921742 (+ 19057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.921743 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.921743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.921751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.923253 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923606 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.923607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.923607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.923662 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923845 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.923846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.923846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.923992 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.924066 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.924226 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.924313 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:08:32.924523 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.899046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.0.0.1:48330)
0807 20:08:32.899515 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.899525 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.899793 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.899978 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.899979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.900323 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.900423 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.900425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.901368 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.901368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.902277 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.902280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.902754 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.902755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.902815 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.921194 (+ 18379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.921199 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.921658 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.923362 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.923363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.923526 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.923724 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.923725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.923781 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.924064 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.924065 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.924111 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.924210 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.924376 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"mutex_wait_us":87,"thread_start_us":359,"threads_started":1}
I20260807 20:08:32.924854 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.914559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.0.0.1:48340)
0807 20:08:32.914625 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.914636 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.914678 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.915195 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.915196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.915368 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.915415 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.915417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.916126 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.916127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.917000 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.917003 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.917350 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.917352 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.917421 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.917750 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.917756 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.919078 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.923504 (+ 4426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923506 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.923506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.923723 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.923927 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.923928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.923992 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.924497 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.924499 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.924545 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.924593 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.924698 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 20:08:32.925220 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.914621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48340 (local address 127.3.15.188:44985)
0807 20:08:32.914870 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.914871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.914877 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:08:32.914953 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.914953 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.914962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.915176 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.915502 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.915989 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.917080 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.917277 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.917326 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.919160 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.919161 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.919162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.919169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.923403 (+ 4234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.923804 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.923805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.923805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.923859 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.924285 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.924286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.924287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.924410 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.924991 (+ 581us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.925036 (+ 45us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.925118 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":188,"threads_started":1}
I20260807 20:08:32.925710 9675 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.15.188:44985: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4df1e015721641e3902f224ee4cd99f5"
member_type: NON_VOTER
last_known_addr {
host: "127.3.15.188"
port: 44985
}
attrs {
promote: true
}
}
dest_uuid: "23cafcb5ec1a429e9291a39af4c9e767"
cas_config_opid_index: 5
I20260807 20:08:32.925990 9675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } }
I20260807 20:08:32.926375 10281 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.926853 10426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:32.927346 10281 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.15.188:44985
I20260807 20:08:32.927415 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.927501 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:32.927942 10426 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Refusing update from remote peer 23cafcb5ec1a429e9291a39af4c9e767: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 20:08:32.928071 10382 master.cc:537] Master@127.3.15.188:44985 shutting down...
I20260807 20:08:32.928217 10280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:08:32.929029 9660 consensus_peers.cc:487] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 -> Peer 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188:44985): Couldn't send request to peer 4df1e015721641e3902f224ee4cd99f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 20:08:32.931348 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.926445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44330)
0807 20:08:32.926617 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.926627 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.926693 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.926919 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.926920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.926921 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:32.926964 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.926966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.928004 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.928005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.930120 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.930137 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.931076 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.931077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.931155 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.931177 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.931229 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.931271 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260807 20:08:32.931459 10382 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:32.931542 10382 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:32.931591 10382 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: stopping tablet replica
I20260807 20:08:32.931597 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.926499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44330 (local address 127.3.15.189:36985)
0807 20:08:32.926543 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.926545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.926664 (+ 119us) server_negotiation.cc:407] Connection header received
0807 20:08:32.926771 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.926772 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.926799 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.926840 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:32.927052 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.927829 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.930246 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.930866 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.930921 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.930998 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.931485 (+ 487us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.931535 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:32.931815 10332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 LEARNER]: Refusing update from remote peer 23cafcb5ec1a429e9291a39af4c9e767: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 20:08:32.932029 10280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 20:08:32.932590 10332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.933696 10484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.933804 10484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: LEARNER
I20260807 20:08:32.934031 10365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23cafcb5ec1a429e9291a39af4c9e767. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: NON_VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: true } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.934119 10365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: LEARNER
I20260807 20:08:32.934584 10280 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: attempting to promote NON_VOTER 68ad022a655a483f9f322e8bb7dfb0c5 to VOTER
I20260807 20:08:32.934814 10280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } }
I20260807 20:08:32.935344 10332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 LEARNER]: Refusing update from remote peer 23cafcb5ec1a429e9291a39af4c9e767: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 20:08:32.935537 10281 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 20:08:32.935622 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.935506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.0.0.1:48342)
0807 20:08:32.935547 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.935587 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44985: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:08:32.935691 9660 consensus_peers.cc:487] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 -> Peer 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188:44985): Couldn't send request to peer 4df1e015721641e3902f224ee4cd99f5. Status: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44985: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 20:08:32.936117 10280 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 68ad022a655a483f9f322e8bb7dfb0c5 (127.3.15.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.936277 10332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 68ad022a655a483f9f322e8bb7dfb0c5 (127.3.15.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.937008 10280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.937090 10280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:32.937142 10280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.937191 10280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:32.937165 10365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.937234 10365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:08:32.946561 10382 master.cc:552] Master@127.3.15.188:44985 shutdown complete.
I20260807 20:08:32.946615 10382 master_runner.cc:305] Clearing existing system tablet
I20260807 20:08:32.946749 10382 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.946797 10382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.947551 10382 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:32.948060 10486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.948197 10382 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.948261 10382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.948297 10382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
uuid: "4df1e015721641e3902f224ee4cd99f5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "3ff34f653bc9cfea00b8549865bf7a1c"
server_key_iv: "adef34277dbd69503991ba7b30fb66b8"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.948983 10382 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:32.950332 10382 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 20:08:32.950392 10382 log.cc:1196] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 20:08:32.950603 10382 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Deleting consensus metadata
I20260807 20:08:32.950736 10382 master_runner.cc:315] Copying system tablet from 127.3.15.190:33055
I20260807 20:08:32.951347 10382 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: tablet copy: Beginning tablet copy session from remote peer at address 127.3.15.190:33055
I20260807 20:08:32.957938 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.951722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.3.15.190:33055)
0807 20:08:32.951772 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:32.951774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:32.951826 (+ 52us) server_negotiation.cc:407] Connection header received
0807 20:08:32.951901 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:32.951901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:32.951909 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:32.951958 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:32.952316 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.952969 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.954110 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.954294 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.954349 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.955366 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:32.955367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:32.955367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:32.955374 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:32.956762 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.957250 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.957251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.957251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.957306 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:32.957501 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:32.957502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:32.957502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:32.957612 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:32.957662 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:32.957812 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:08:32.957876 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:08:32.958810 10492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:32.951634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:53782)
0807 20:08:32.951757 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:32.951769 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:32.951839 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:32.951965 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:32.951966 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:32.952165 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:32.952240 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.952242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.953048 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.953049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.954031 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:32.954034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:32.954365 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:32.954366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:32.954437 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:32.954894 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:32.954899 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:32.955285 (+ 386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:32.956958 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.956959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.956960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.957183 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.957369 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:32.957370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:32.957370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:32.957436 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:32.957668 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:32.957668 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:32.957715 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:08:32.957788 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:08:32.957890 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":78,"threads_started":1}
I20260807 20:08:32.959087 9697 tablet_copy_service.cc:140] P 23cafcb5ec1a429e9291a39af4c9e767: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4df1e015721641e3902f224ee4cd99f5 ({username='oryx', principal='oryx/127.3.15.188@KRBTEST.COM'} at 127.0.0.1:53782)
I20260807 20:08:32.959152 9697 tablet_copy_service.cc:161] P 23cafcb5ec1a429e9291a39af4c9e767: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4df1e015721641e3902f224ee4cd99f5 at {username='oryx', principal='oryx/127.3.15.188@KRBTEST.COM'} at 127.0.0.1:53782: session id = 4df1e015721641e3902f224ee4cd99f5-00000000000000000000000000000000
I20260807 20:08:32.959733 9697 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 20:08:32.960219 10382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 20:08:32.961253 10382 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: tablet copy: Starting download of 0 data blocks...
I20260807 20:08:32.961408 10382 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: tablet copy: Starting download of 1 WAL segments...
I20260807 20:08:32.962208 10382 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 20:08:32.962946 9697 tablet_copy_service.cc:342] P 23cafcb5ec1a429e9291a39af4c9e767: Request end of tablet copy session 4df1e015721641e3902f224ee4cd99f5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.15.188@KRBTEST.COM'} at 127.0.0.1:53782
I20260807 20:08:32.963006 9697 tablet_copy_service.cc:433] P 23cafcb5ec1a429e9291a39af4c9e767: ending tablet copy session 4df1e015721641e3902f224ee4cd99f5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4df1e015721641e3902f224ee4cd99f5
I20260807 20:08:32.965708 10382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:32.966889 10496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.967031 10493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:32.967315 10494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:32.967398 10382 server_base.cc:1034] running on GCE node
I20260807 20:08:32.967545 10382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:32.967689 10382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:32.968822 10382 hybrid_clock.cc:648] HybridClock initialized: now 1786133312968815 us; error 24 us; skew 500 ppm
Aug 07 20:08:32 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133312, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:32.970221 10382 init.cc:377] Logged in from keytab as oryx/127.3.15.188@KRBTEST.COM (short username oryx)
I20260807 20:08:32.970492 10382 webserver.cc:460] Webserver started at http://127.3.15.188:36999/ using document root <none> and password file <none>
I20260807 20:08:32.970619 10382 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:32.970655 10382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:32.971357 10382 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:32.971889 10503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.972030 10382 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:32.972077 10382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
uuid: "4df1e015721641e3902f224ee4cd99f5"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "3ff34f653bc9cfea00b8549865bf7a1c"
server_key_iv: "adef34277dbd69503991ba7b30fb66b8"
server_key_version: "encryptionkey@0"
I20260807 20:08:32.972138 10382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:32.985533 10382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:32.987309 10382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:32.992744 10382 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.188:44985
I20260807 20:08:32.992857 10557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.188:44985 every 8 connection(s)
I20260807 20:08:32.993113 10382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
I20260807 20:08:32.994210 10559 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:08:32.994895 10559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Bootstrap starting.
I20260807 20:08:32.998525 10559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:08:32.998811 10559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Bootstrap complete.
I20260807 20:08:32.999048 10559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } }
I20260807 20:08:32.999147 10559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4df1e015721641e3902f224ee4cd99f5, State: Initialized, Role: LEARNER
I20260807 20:08:32.999189 10559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } }
I20260807 20:08:32.999370 10559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:32.999385 10562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:32.999537 10562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: This master's current role is: LEARNER
I20260807 20:08:33.002432 10573 catalog_manager.cc:1259] Loaded cluster ID: 162a9861a0894381897d0107e34039b0
I20260807 20:08:33.002485 10573 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: loading cluster ID for follower catalog manager: success
I20260807 20:08:33.003201 10573 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: acquiring CA information for follower catalog manager: success
I20260807 20:08:33.003479 10573 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:08:33.042636 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.035686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.3.15.188:44985)
0807 20:08:33.035842 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.035844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.035851 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:33.035876 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.035877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.035887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.035961 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.036538 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.037257 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.037977 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.038250 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.038321 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.039831 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.039833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.039833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.039842 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.041418 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.041795 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.041797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.041797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.041859 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.042068 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.042069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.042069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.042185 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.042283 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.042428 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.042523 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260807 20:08:33.043324 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9743
I20260807 20:08:33.051952 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:40975
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:33055
--webserver_port=44527
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:33.136309 10575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:33.136549 10575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:33.136574 10575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:33.137748 10575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:33.137790 10575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:33.137815 10575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:33.139271 10575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:40975
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=44527
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:33.139452 10575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:33.139628 10575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:33.144533 10581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:33.144943 10583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:33.145064 10585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:33.145285 10575 server_base.cc:1034] running on GCE node
I20260807 20:08:33.145504 10575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:33.145738 10575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:33.146888 10575 hybrid_clock.cc:648] HybridClock initialized: now 1786133313146882 us; error 33 us; skew 500 ppm
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:33.149520 10575 init.cc:377] Logged in from keytab as oryx/127.3.15.129@KRBTEST.COM (short username oryx)
I20260807 20:08:33.149941 10575 webserver.cc:460] Webserver started at http://127.3.15.129:44527/ using document root <none> and password file <none>
I20260807 20:08:33.150193 10575 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:33.150278 10575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:33.151595 10575 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:33.152153 10593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.152335 10575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:33.152428 10575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "eac1c7997cf5407b91acb8d2c8d15500"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "445d86a772a302b187866a9bcd1d7752"
server_key_iv: "6d7757a6ed68aff007117b0ad7301588"
server_key_version: "encryptionkey@0"
I20260807 20:08:33.152889 10575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:33.161923 10575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:33.164251 10575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:33.164361 10575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:33.164585 10575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:33.164892 10575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:33.164949 10575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.164976 10575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:33.165002 10575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.171438 10575 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40975
I20260807 20:08:33.171540 10714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40975 every 8 connection(s)
I20260807 20:08:33.171836 10575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:33.177500 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 10575
I20260807 20:08:33.177657 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9887
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.188@KRBTEST.COM
I20260807 20:08:33.185988 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.173039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:37247 (local address 127.3.15.190:33055)
0807 20:08:33.173113 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.173115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.173126 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:33.174304 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.174305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.174320 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.174385 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.175301 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.176103 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177236 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.177795 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177854 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.181832 (+ 3978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.181833 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.181834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.181841 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.184285 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.184898 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.184900 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.184900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.184962 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185253 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.185254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.185254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.185383 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.185439 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.185809 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.185891 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 20:08:33.185988 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.173287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:43803 (local address 127.3.15.189:36985)
0807 20:08:33.173363 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.173365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.174560 (+ 1195us) server_negotiation.cc:407] Connection header received
0807 20:08:33.174683 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.174684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.174695 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.174744 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.175275 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.175765 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.176763 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.177000 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177081 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.181092 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.181094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.181094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.181102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.183147 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.184972 (+ 1825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.184973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.184973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.185028 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185253 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.185253 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.185254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.185395 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.185444 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.185809 (+ 365us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.185894 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:08:33.186059 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.172852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:60307 (local address 127.3.15.188:44985)
0807 20:08:33.172930 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.172932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.173175 (+ 243us) server_negotiation.cc:407] Connection header received
0807 20:08:33.174321 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.174322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.174338 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.174393 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.175302 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.176059 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177263 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.177581 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177662 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.182212 (+ 4550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.182214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.182214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.182225 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.184442 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185060 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.185061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.185062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.185142 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185364 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.185365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.185365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.185505 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.185559 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.185803 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.185908 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:08:33.186661 10719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.172572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:37247)
0807 20:08:33.172760 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.172823 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.174172 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.174448 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.174452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.175067 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.175191 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.175197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.176189 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.176189 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.177148 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.177150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.177904 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.177957 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.178427 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.178436 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.181724 (+ 3288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.184395 (+ 2671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.184395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.184396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.184804 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.185045 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.185046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.185174 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.185476 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.185483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.185645 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.186171 (+ 526us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.186394 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":65,"thread_start_us":99,"threads_started":1}
I20260807 20:08:33.186661 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.172599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.3.15.129:43803)
0807 20:08:33.174370 (+ 1771us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.174380 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.174609 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.174748 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.174749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.174964 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.175189 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.175194 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.175868 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.175869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.176673 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.176675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.177089 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.177249 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.178707 (+ 1458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.178712 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.180978 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.183608 (+ 2630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.183616 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.183618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.184889 (+ 1271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.185103 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185103 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.185104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.185174 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.185476 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.185483 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.185646 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.186156 (+ 510us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.186502 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1753,"mutex_wait_us":171,"thread_start_us":98,"threads_started":1}
I20260807 20:08:33.187029 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.172572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.3.15.129:60307)
0807 20:08:33.172854 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.172865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.174200 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.174472 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.174472 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.174938 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.175191 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.175197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.176159 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.176160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.177148 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.177150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.177667 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.177669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.179246 (+ 1577us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.179634 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.179640 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.182091 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.184540 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.184540 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.184541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.184968 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.185226 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.185226 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.185227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.185295 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.185590 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.185591 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.185657 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.186432 (+ 775us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.186580 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":127,"spinlock_wait_cycles":7936,"thread_start_us":103,"threads_started":1}
I20260807 20:08:33.187748 10717 heartbeater.cc:347] Connected to a master server at 127.3.15.189:36985
I20260807 20:08:33.187760 10715 heartbeater.cc:347] Connected to a master server at 127.3.15.188:44985
I20260807 20:08:33.187870 10717 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.187871 10715 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.188068 10717 heartbeater.cc:511] Master 127.3.15.189:36985 requested a full tablet report, sending...
I20260807 20:08:33.188076 10715 heartbeater.cc:511] Master 127.3.15.188:44985 requested a full tablet report, sending...
I20260807 20:08:33.188560 10321 ts_manager.cc:194] Registered new tserver with Master: eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975)
I20260807 20:08:33.188657 10520 ts_manager.cc:194] Registered new tserver with Master: eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975)
I20260807 20:08:33.188815 10716 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33055
I20260807 20:08:33.188867 10716 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.188933 10716 heartbeater.cc:511] Master 127.3.15.190:33055 requested a full tablet report, sending...
I20260807 20:08:33.189225 9675 ts_manager.cc:194] Re-registered known tserver with Master: eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975)
I20260807 20:08:33.189713 9675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:37247
I20260807 20:08:33.191751 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:45133
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:33055
--webserver_port=42603
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:33.276624 10722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:33.276841 10722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:33.276865 10722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:33.278004 10722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:33.278045 10722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:33.278071 10722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:33.279546 10722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:45133
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=42603
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:33.279788 10722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:33.280032 10722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:33.285431 10727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:33.285415 10728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:33.285463 10730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:33.285403 10722 server_base.cc:1034] running on GCE node
I20260807 20:08:33.285844 10722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:33.286075 10722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:33.287303 10722 hybrid_clock.cc:648] HybridClock initialized: now 1786133313287289 us; error 30 us; skew 500 ppm
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:33.290041 10722 init.cc:377] Logged in from keytab as oryx/127.3.15.130@KRBTEST.COM (short username oryx)
I20260807 20:08:33.290482 10722 webserver.cc:460] Webserver started at http://127.3.15.130:42603/ using document root <none> and password file <none>
I20260807 20:08:33.290704 10722 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:33.290786 10722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:33.292188 10722 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:33.292810 10737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.293018 10722 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:33.293118 10722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "e286b497d371491ba4c552c0f9eb285b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "493d08a5f97829be3bc840e47eac7763"
server_key_iv: "4fcb5ae99510c518d282b77f7b77318f"
server_key_version: "encryptionkey@0"
I20260807 20:08:33.293479 10722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:33.308148 10722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:33.310475 10722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:33.310632 10722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:33.310850 10722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:33.311167 10722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:33.311219 10722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.311259 10722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:33.311286 10722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.317240 10722 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:45133
I20260807 20:08:33.317284 10855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:45133 every 8 connection(s)
I20260807 20:08:33.317732 10722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:33.321225 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.317466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.0.0.1:48364)
0807 20:08:33.317529 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.317543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.317607 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.317782 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.317784 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.317786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:33.317845 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.317848 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.318804 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.318805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.320074 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.320078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.321006 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.321007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.321070 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.321074 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.321114 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.321156 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:08:33.321225 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.317587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48364 (local address 127.3.15.188:44985)
0807 20:08:33.317633 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.317635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.317651 (+ 16us) server_negotiation.cc:407] Connection header received
0807 20:08:33.317678 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.317678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.317698 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.317754 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:33.317922 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.318660 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.320172 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.320924 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.321005 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.321086 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.321119 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.321156 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:08:33.321521 10531 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 20:08:33.321733 10562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23cafcb5ec1a429e9291a39af4c9e767. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:33.321856 10562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: This master's current role is: LEARNER
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.188@KRBTEST.COM
I20260807 20:08:33.327138 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 10722
I20260807 20:08:33.327315 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 10037
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:33.330505 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.320589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:41103 (local address 127.3.15.189:36985)
0807 20:08:33.320638 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.320640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.322277 (+ 1637us) server_negotiation.cc:407] Connection header received
0807 20:08:33.322336 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.322337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.322348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.322404 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.322756 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.323223 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.323996 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.324202 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324283 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.326727 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.326728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.326729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.326735 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.328477 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.329041 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.329043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.329043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.329118 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.329526 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.329528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.329528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.329686 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.329767 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.330197 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.330342 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:08:33.331466 10863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.320525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.3.15.130:41103)
0807 20:08:33.322142 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.322152 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.322284 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.322409 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.322409 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.322571 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.322641 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.322643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.323307 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.323308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.323906 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.323908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.324297 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.324433 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.324961 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.324971 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.326636 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.328564 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.328572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.328574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.328934 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.329215 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.329216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.329217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.329398 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.329762 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.329768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.329990 (+ 222us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.330733 (+ 743us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.331059 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":76,"threads_started":1}
I20260807 20:08:33.331880 10860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.318461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.3.15.130:53683)
0807 20:08:33.318637 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.318679 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.320629 (+ 1950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.321355 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.321355 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.321567 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.321733 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.321739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.322720 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.322720 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.323914 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.323916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.324420 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.324486 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.325207 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.325211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.327146 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.330652 (+ 3506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.330653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.330653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.330955 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.331179 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.331180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.331180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.331257 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.331561 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.331562 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.331604 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.331668 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.331772 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":49,"thread_start_us":103,"threads_started":1}
I20260807 20:08:33.332171 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.318693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:48217 (local address 127.3.15.190:33055)
0807 20:08:33.318739 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.318741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.318979 (+ 238us) server_negotiation.cc:407] Connection header received
0807 20:08:33.320465 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.320466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.320478 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.320507 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.321820 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.322480 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324023 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.324316 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324358 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.328368 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.328370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.328370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.328378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.330563 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.331174 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.331175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.331176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.331245 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.331457 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.331457 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.331458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.331589 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.331634 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.331972 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.332075 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:08:33.332329 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.318695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:53683 (local address 127.3.15.188:44985)
0807 20:08:33.320250 (+ 1555us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.320253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.320263 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:33.320687 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.320688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.320699 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.320994 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.321838 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.322647 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324017 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.324317 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324370 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.327657 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.327659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.327659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.327667 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.330554 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.331041 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.331043 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.331043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.331108 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.331329 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.331330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.331330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.331476 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.331536 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.332154 (+ 618us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.332241 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1526,"thread_start_us":70,"threads_started":1}
I20260807 20:08:33.332682 10861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.318465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:48217)
0807 20:08:33.318806 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.318816 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.320340 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.320685 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.320689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.321147 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.321733 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.321739 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.322595 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.322596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.323914 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.323916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.324448 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.324449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.324507 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.325298 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.325303 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.328281 (+ 2978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.330776 (+ 2495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.330777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.330777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.331085 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.331330 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.331331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.331331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.331383 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.331795 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.331795 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.331833 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.332519 (+ 686us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.332617 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":57,"thread_start_us":91,"threads_started":1}
I20260807 20:08:33.333158 10857 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33055
I20260807 20:08:33.333290 10857 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.333568 10857 heartbeater.cc:511] Master 127.3.15.190:33055 requested a full tablet report, sending...
I20260807 20:08:33.334007 9675 ts_manager.cc:194] Re-registered known tserver with Master: e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133)
I20260807 20:08:33.334373 9675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:48217
I20260807 20:08:33.337455 10856 heartbeater.cc:347] Connected to a master server at 127.3.15.188:44985
I20260807 20:08:33.337543 10856 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.337635 10856 heartbeater.cc:511] Master 127.3.15.188:44985 requested a full tablet report, sending...
I20260807 20:08:33.337978 10520 ts_manager.cc:194] Registered new tserver with Master: e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133)
I20260807 20:08:33.338079 10858 heartbeater.cc:347] Connected to a master server at 127.3.15.189:36985
I20260807 20:08:33.338140 10858 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.338198 10858 heartbeater.cc:511] Master 127.3.15.189:36985 requested a full tablet report, sending...
I20260807 20:08:33.338497 10321 ts_manager.cc:194] Registered new tserver with Master: e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133)
I20260807 20:08:33.341797 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:42541
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:33055
--webserver_port=33279
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:33.426853 10864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:33.427062 10864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:33.427085 10864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:33.428166 10864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:33.428200 10864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:33.428225 10864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:33.429730 10864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:42541
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=33279
--tserver_master_addrs=127.3.15.190:33055,127.3.15.189:36985,127.3.15.188:44985
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:33.429970 10864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:33.430173 10864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:33.435124 10870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:33.435144 10869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:33.435174 10872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:33.435727 10864 server_base.cc:1034] running on GCE node
I20260807 20:08:33.435941 10864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:33.436190 10864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:33.437366 10864 hybrid_clock.cc:648] HybridClock initialized: now 1786133313437344 us; error 35 us; skew 500 ppm
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:33.439898 10864 init.cc:377] Logged in from keytab as oryx/127.3.15.131@KRBTEST.COM (short username oryx)
I20260807 20:08:33.440351 10864 webserver.cc:460] Webserver started at http://127.3.15.131:33279/ using document root <none> and password file <none>
I20260807 20:08:33.440567 10864 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:33.440640 10864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:33.442066 10864 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:33.442662 10879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.442855 10864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:33.442975 10864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "52da89c27ef146a19f5355741440587b"
format_stamp: "Formatted at 2026-08-07 20:08:32 on dist-test-slave-1x4g"
server_key: "bf2ce823608789162f5f5ce3cbe5c15a"
server_key_iv: "4eb7ce9633ae6a9e4d20bb3e32e54314"
server_key_version: "encryptionkey@0"
I20260807 20:08:33.443317 10864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:33.465277 10864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:33.467218 10864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:33.467360 10864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:33.467600 10864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:33.467924 10864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:33.467967 10864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.468007 10864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:33.468034 10864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:33.475389 10864 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:42541
I20260807 20:08:33.475446 10999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:42541 every 8 connection(s)
I20260807 20:08:33.475798 10864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:33.477427 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 10864
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.189@KRBTEST.COM
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.188@KRBTEST.COM
I20260807 20:08:33.487047 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.478857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.3.15.188:44985)
0807 20:08:33.478895 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.478897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.479186 (+ 289us) server_negotiation.cc:407] Connection header received
0807 20:08:33.479296 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.479297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.479307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.479338 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.480182 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.480851 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.481937 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.482239 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.482297 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.484109 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.484111 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.484111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.484118 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.485759 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.486105 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.486106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.486107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.486165 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.486401 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.486402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.486403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.486565 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.486644 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.486859 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.486944 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 20:08:33 dist-test-slave-1x4g krb5kdc[9627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133313, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:33.490219 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.477293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:35225 (local address 127.3.15.189:36985)
0807 20:08:33.477335 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.477337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.479778 (+ 2441us) server_negotiation.cc:407] Connection header received
0807 20:08:33.479878 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.479879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.479890 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.479919 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.480283 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.480977 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.481871 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.482176 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.482240 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.485852 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.485854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.485854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.485861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.488461 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489071 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.489072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.489072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.489130 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489467 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.489469 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.489469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.489599 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.489648 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.490038 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.490115 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:33.490732 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.476637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.3.15.131:45759)
0807 20:08:33.476829 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.476871 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.478333 (+ 1462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.478586 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.478591 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.479244 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.479575 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.479583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.480613 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.480614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.481835 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.481837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.482489 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.482490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.482565 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.483523 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.483528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.486352 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.489265 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489266 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.489267 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.489506 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.489725 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.489726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.489786 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.490125 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.490126 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.490167 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.490231 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.490409 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":4,"thread_start_us":91,"threads_started":1}
I20260807 20:08:33.490935 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.477030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.3.15.131:35225)
0807 20:08:33.479581 (+ 2551us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.479592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.479808 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.479976 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.479977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.480140 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.480200 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.480202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.481050 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.481051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.481779 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.481782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.482309 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.482310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.482458 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.483181 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.483191 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.485776 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.488550 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.488558 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.488560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.488979 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.489208 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.489209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.489364 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.489703 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.489711 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.489872 (+ 161us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.490336 (+ 464us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.490489 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2534,"mutex_wait_us":26,"thread_start_us":79,"threads_started":1}
I20260807 20:08:33.491472 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.476968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:45759 (local address 127.3.15.188:44985)
0807 20:08:33.477044 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.477053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.477116 (+ 63us) server_negotiation.cc:407] Connection header received
0807 20:08:33.478425 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.478427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.478442 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.478505 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.479705 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.480488 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.481927 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.482353 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.482407 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.486454 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.486455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.486456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.486463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.489172 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489591 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.489592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.489592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.489653 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.489849 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.489850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.489850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.490026 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.490085 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.490674 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.490770 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:08:33.491851 11000 heartbeater.cc:347] Connected to a master server at 127.3.15.188:44985
I20260807 20:08:33.491967 11000 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.492173 11000 heartbeater.cc:511] Master 127.3.15.188:44985 requested a full tablet report, sending...
I20260807 20:08:33.492259 11002 heartbeater.cc:347] Connected to a master server at 127.3.15.189:36985
I20260807 20:08:33.492337 11002 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.492414 11002 heartbeater.cc:511] Master 127.3.15.189:36985 requested a full tablet report, sending...
I20260807 20:08:33.492626 10520 ts_manager.cc:194] Registered new tserver with Master: 52da89c27ef146a19f5355741440587b (127.3.15.131:42541)
I20260807 20:08:33.492740 10321 ts_manager.cc:194] Registered new tserver with Master: 52da89c27ef146a19f5355741440587b (127.3.15.131:42541)
I20260807 20:08:33.493467 9737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.477173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:44709 (local address 127.3.15.190:33055)
0807 20:08:33.477223 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.477224 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.477233 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:33.478472 (+ 1239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.478473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.478486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.478520 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.479766 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.480541 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.487158 (+ 6617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.487480 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.487523 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.490340 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.490342 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.490342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.490349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.492158 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.492643 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.492645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.492645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.492707 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.492951 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.492952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.492953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.493065 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.493136 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.493305 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.493376 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:08:33.493484 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.476637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:44709)
0807 20:08:33.476832 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.476872 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.478326 (+ 1454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.478596 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.478597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.479234 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:33.479575 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.479583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.480634 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.480635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.485669 (+ 5034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.485672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.488122 (+ 2450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.488123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.488184 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.488686 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:33.488692 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:33.490234 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:33.492253 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.492254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.492254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.492557 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.492792 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:33.492793 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:33.492793 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:33.492854 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:33.493143 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:33.493144 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:33.493196 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.493264 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.493366 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":3334,"thread_start_us":115,"threads_started":1}
I20260807 20:08:33.494163 11001 heartbeater.cc:347] Connected to a master server at 127.3.15.190:33055
I20260807 20:08:33.494216 11001 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:33.494279 11001 heartbeater.cc:511] Master 127.3.15.190:33055 requested a full tablet report, sending...
I20260807 20:08:33.494489 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.479266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.3.15.189:36985)
0807 20:08:33.480851 (+ 1585us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.480853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.482248 (+ 1395us) server_negotiation.cc:407] Connection header received
0807 20:08:33.482273 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.482274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.482286 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.482411 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.483208 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.483970 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.485079 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.485447 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.485507 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.487227 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.487228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.487228 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.487235 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.492468 (+ 5233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.493258 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.493259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.493259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.493320 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.493499 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.493499 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.493500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.493626 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.493713 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.494276 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.494400 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1566,"thread_start_us":71,"threads_started":1}
I20260807 20:08:33.494503 9675 ts_manager.cc:194] Re-registered known tserver with Master: 52da89c27ef146a19f5355741440587b (127.3.15.131:42541)
I20260807 20:08:33.494879 9675 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:44709
I20260807 20:08:33.498036 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.478691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.3.15.190:33055)
0807 20:08:33.488807 (+ 10116us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.488809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.488817 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:33.488840 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.488841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.488853 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.489028 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.489676 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.490462 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.493213 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.493548 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.494100 (+ 552us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.495711 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.495712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.495712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.495718 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.497171 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.497502 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.497503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.497503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.497553 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.497697 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.497697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.497697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.497779 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.497845 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.497919 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.497976 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10064,"thread_start_us":55,"threads_started":1}
I20260807 20:08:33.499660 9675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53794:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:33.501494 9675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:33.502424 11021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: NON_VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: true } } }
I20260807 20:08:33.502521 11021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:33.503652 11021 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: attempting to promote NON_VOTER 4df1e015721641e3902f224ee4cd99f5 to VOTER
I20260807 20:08:33.503779 11021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } }
I20260807 20:08:33.504185 10530 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 LEARNER]: Refusing update from remote peer 23cafcb5ec1a429e9291a39af4c9e767: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 20:08:33.504369 10281 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 20:08:33.504452 10332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Refusing update from remote peer 23cafcb5ec1a429e9291a39af4c9e767: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 20:08:33.504632 10281 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 20:08:33.505280 11020 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.505488 10332 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.505475 10531 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.506317 11021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.506399 11021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:33.506345 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.506439 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:33.506541 10562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.506635 10562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:08:33.506774 11030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.506778 10281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "23cafcb5ec1a429e9291a39af4c9e767" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:33.506850 11030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:08:33.506858 10281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:33.511567 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.507579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40975 (local address 127.0.0.1:57486)
0807 20:08:33.507724 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.507736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.507781 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.508162 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.508163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.508165 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:33.508212 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.508214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.509817 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.509818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.510571 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.510573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.511347 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.511349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.511408 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.511413 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.511455 (+ 42us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.511495 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":103,"threads_started":1}
I20260807 20:08:33.512142 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.507695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.3.15.129:40975)
0807 20:08:33.507893 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.507899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.507910 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:33.507953 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.507955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.507973 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.508159 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:33.508282 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.509733 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.510669 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.511247 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.511827 (+ 580us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.511921 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.511953 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.512047 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260807 20:08:33.512260 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.507520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42541 (local address 127.0.0.1:52850)
0807 20:08:33.507725 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.507736 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.507972 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.508523 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.508525 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.508528 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:33.508569 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.508571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.509873 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.509874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.511080 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.511082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.511979 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.511981 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.512079 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.512084 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.512145 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.512188 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":33}
I20260807 20:08:33.512428 11036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.507912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:45133 (local address 127.0.0.1:52154)
0807 20:08:33.508375 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.508383 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.508580 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.509036 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.509037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.509039 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:33.509096 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.509098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.509946 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.509947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.511071 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.511074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.512156 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.512157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.512254 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.512259 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.512314 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.512355 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":4,"thread_start_us":209,"threads_started":1}
I20260807 20:08:33.512476 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.507605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.3.15.131:42541)
0807 20:08:33.508292 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.508296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.508305 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:33.508345 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.508347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.508361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.508655 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:33.508719 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.509811 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.511222 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.511890 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.511976 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.512065 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.512162 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.512204 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":95,"threads_started":1}
I20260807 20:08:33.512706 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.508001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.3.15.130:45133)
0807 20:08:33.508333 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.508337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.508576 (+ 239us) server_negotiation.cc:407] Connection header received
0807 20:08:33.508705 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.508707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.508720 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.509040 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:33.509155 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.509879 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.511162 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.511949 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.512027 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.512115 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.512575 (+ 460us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.512621 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":112,"threads_started":1}
I20260807 20:08:33.513181 10930 tablet_service.cc:1467] Processing CreateTablet for tablet 1f49654dd556416bb82db229ed4ad441 (DEFAULT_TABLE table=test-table [id=85456cb4fcd8452d9fee87b24a2b3ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:33.513243 10644 tablet_service.cc:1467] Processing CreateTablet for tablet 1f49654dd556416bb82db229ed4ad441 (DEFAULT_TABLE table=test-table [id=85456cb4fcd8452d9fee87b24a2b3ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:33.513513 10930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f49654dd556416bb82db229ed4ad441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:33.513460 10788 tablet_service.cc:1467] Processing CreateTablet for tablet 1f49654dd556416bb82db229ed4ad441 (DEFAULT_TABLE table=test-table [id=85456cb4fcd8452d9fee87b24a2b3ed1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:33.513629 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f49654dd556416bb82db229ed4ad441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:33.513764 10788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f49654dd556416bb82db229ed4ad441. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:33.516734 11037 tablet_bootstrap.cc:492] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Bootstrap starting.
I20260807 20:08:33.517452 11038 tablet_bootstrap.cc:492] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Bootstrap starting.
I20260807 20:08:33.518083 11038 tablet_bootstrap.cc:654] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:33.518083 11037 tablet_bootstrap.cc:654] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:33.518478 11037 log.cc:824] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:33.518500 11038 log.cc:824] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:33.518832 11039 tablet_bootstrap.cc:492] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Bootstrap starting.
I20260807 20:08:33.519372 11038 tablet_bootstrap.cc:492] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: No bootstrap required, opened a new log
I20260807 20:08:33.519454 11038 ts_tablet_manager.cc:1397] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:33.519479 11039 tablet_bootstrap.cc:654] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:33.519665 11037 tablet_bootstrap.cc:492] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: No bootstrap required, opened a new log
I20260807 20:08:33.519749 11037 ts_tablet_manager.cc:1397] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:08:33.519910 11039 log.cc:824] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:33.520706 11039 tablet_bootstrap.cc:492] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: No bootstrap required, opened a new log
I20260807 20:08:33.520810 11039 ts_tablet_manager.cc:1397] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:33.521780 11037 raft_consensus.cc:348] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.521796 11038 raft_consensus.cc:348] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.521974 11037 raft_consensus.cc:374] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:33.521981 11038 raft_consensus.cc:374] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:33.522009 11037 raft_consensus.cc:729] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52da89c27ef146a19f5355741440587b, State: Initialized, Role: FOLLOWER
I20260807 20:08:33.522027 11038 raft_consensus.cc:729] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286b497d371491ba4c552c0f9eb285b, State: Initialized, Role: FOLLOWER
I20260807 20:08:33.522161 11037 consensus_queue.cc:260] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.522161 11038 consensus_queue.cc:260] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.522598 11038 ts_tablet_manager.cc:1428] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:08:33.522858 10857 heartbeater.cc:503] Master 127.3.15.190:33055 was elected leader, sending a full tablet report...
I20260807 20:08:33.523406 11039 raft_consensus.cc:348] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.523568 11039 raft_consensus.cc:374] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:33.523622 11039 raft_consensus.cc:729] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac1c7997cf5407b91acb8d2c8d15500, State: Initialized, Role: FOLLOWER
I20260807 20:08:33.523727 11039 consensus_queue.cc:260] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.524097 10716 heartbeater.cc:503] Master 127.3.15.190:33055 was elected leader, sending a full tablet report...
I20260807 20:08:33.524036 11039 ts_tablet_manager.cc:1428] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:08:33.524886 11001 heartbeater.cc:503] Master 127.3.15.190:33055 was elected leader, sending a full tablet report...
I20260807 20:08:33.525183 11037 ts_tablet_manager.cc:1428] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 20:08:33.536263 11045 raft_consensus.cc:482] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:33.536375 11045 raft_consensus.cc:504] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.536743 11045 leader_election.cc:288] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52da89c27ef146a19f5355741440587b (127.3.15.131:42541), e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133)
I20260807 20:08:33.539642 10720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.536953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:45133 (local address 127.3.15.129:54659)
0807 20:08:33.537140 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.537151 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.537200 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.537327 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.537328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.537331 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:33.537456 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.537458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.538085 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.538085 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.538826 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.538828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.539385 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.539386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.539467 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.539470 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.539515 (+ 45us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.539561 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":148}
I20260807 20:08:33.539655 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.537039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:54659 (local address 127.3.15.130:45133)
0807 20:08:33.537081 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.537083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.537205 (+ 122us) server_negotiation.cc:407] Connection header received
0807 20:08:33.537253 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.537254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.537266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.537320 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:33.537533 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.538004 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.538911 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.539307 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.539382 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.539392 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.539534 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.539580 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:08:33.539696 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.536969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:56903 (local address 127.3.15.131:42541)
0807 20:08:33.537021 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.537023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.537046 (+ 23us) server_negotiation.cc:407] Connection header received
0807 20:08:33.537124 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.537125 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.537139 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.537196 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:33.537373 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.537912 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.538980 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.539316 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.539399 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.539407 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.539583 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.539625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:08:33.539748 10721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.536953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42541 (local address 127.3.15.129:56903)
0807 20:08:33.536995 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:33.537008 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:33.537043 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:33.537222 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:33.537223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:33.537225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:33.537304 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.537306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.538022 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.538023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.538851 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:33.538854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.539417 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.539418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:33.539493 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.539510 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 20:08:33.539569 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:08:33.539605 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":49}
I20260807 20:08:33.540060 10808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f49654dd556416bb82db229ed4ad441" candidate_uuid: "eac1c7997cf5407b91acb8d2c8d15500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286b497d371491ba4c552c0f9eb285b" is_pre_election: true
I20260807 20:08:33.540212 10808 raft_consensus.cc:2454] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac1c7997cf5407b91acb8d2c8d15500 in term 0.
I20260807 20:08:33.540326 10952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f49654dd556416bb82db229ed4ad441" candidate_uuid: "eac1c7997cf5407b91acb8d2c8d15500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52da89c27ef146a19f5355741440587b" is_pre_election: true
I20260807 20:08:33.540468 10952 raft_consensus.cc:2454] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac1c7997cf5407b91acb8d2c8d15500 in term 0.
I20260807 20:08:33.540444 10595 leader_election.cc:302] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e286b497d371491ba4c552c0f9eb285b, eac1c7997cf5407b91acb8d2c8d15500; no voters:
I20260807 20:08:33.540593 11045 raft_consensus.cc:2781] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:33.540642 11045 raft_consensus.cc:482] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:33.540673 11045 raft_consensus.cc:3037] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:33.541510 11045 raft_consensus.cc:504] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.541674 11045 leader_election.cc:288] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [CANDIDATE]: Term 1 election: Requested vote from peers 52da89c27ef146a19f5355741440587b (127.3.15.131:42541), e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133)
I20260807 20:08:33.541802 10808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f49654dd556416bb82db229ed4ad441" candidate_uuid: "eac1c7997cf5407b91acb8d2c8d15500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e286b497d371491ba4c552c0f9eb285b"
I20260807 20:08:33.541806 10952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f49654dd556416bb82db229ed4ad441" candidate_uuid: "eac1c7997cf5407b91acb8d2c8d15500" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52da89c27ef146a19f5355741440587b"
I20260807 20:08:33.541879 10808 raft_consensus.cc:3037] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:33.541906 10952 raft_consensus.cc:3037] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:33.542564 10808 raft_consensus.cc:2454] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac1c7997cf5407b91acb8d2c8d15500 in term 1.
I20260807 20:08:33.542682 10952 raft_consensus.cc:2454] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac1c7997cf5407b91acb8d2c8d15500 in term 1.
I20260807 20:08:33.542721 10595 leader_election.cc:302] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e286b497d371491ba4c552c0f9eb285b, eac1c7997cf5407b91acb8d2c8d15500; no voters:
I20260807 20:08:33.542821 11045 raft_consensus.cc:2781] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:33.542979 11045 raft_consensus.cc:686] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 1 LEADER]: Becoming Leader. State: Replica: eac1c7997cf5407b91acb8d2c8d15500, State: Running, Role: LEADER
I20260807 20:08:33.543088 11045 consensus_queue.cc:237] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } }
I20260807 20:08:33.544134 9675 catalog_manager.cc:5495] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "eac1c7997cf5407b91acb8d2c8d15500" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } health_report { overall_health: UNKNOWN } } }
I20260807 20:08:33.555008 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.552805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57498 (local address 127.3.15.129:40975)
0807 20:08:33.552864 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.552865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.552873 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:33.552946 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.552946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.552964 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.553023 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:33.553142 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.553614 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.554305 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.554503 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.554556 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.554690 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:33.554785 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:33.554861 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.554902 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.554924 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:08:33.556885 10808 raft_consensus.cc:1261] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Refusing update from remote peer eac1c7997cf5407b91acb8d2c8d15500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:33.556915 10952 raft_consensus.cc:1261] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Refusing update from remote peer eac1c7997cf5407b91acb8d2c8d15500: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:33.557256 11045 consensus_queue.cc:1035] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:33.557389 11046 consensus_queue.cc:1035] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:33.561164 11050 mvcc.cc:205] Tried to move back new op lower bound from 7316002052326383616 to 7316002052273225728. Current Snapshot: MvccSnapshot[applied={T|T < 7316002052326383616}]
I20260807 20:08:33.563650 9675 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:33.563741 9675 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53794:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:33.565241 9675 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 23cafcb5ec1a429e9291a39af4c9e767: Sending DeleteTablet for 3 replicas of tablet 1f49654dd556416bb82db229ed4ad441
I20260807 20:08:33.565539 10930 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f49654dd556416bb82db229ed4ad441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:33 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:52850
I20260807 20:08:33.565659 10788 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f49654dd556416bb82db229ed4ad441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:33 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:52154
I20260807 20:08:33.565974 11060 tablet_replica.cc:330] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: stopping tablet replica
I20260807 20:08:33.566087 11060 raft_consensus.cc:2229] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:33.566182 11060 raft_consensus.cc:2258] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:33.566190 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f49654dd556416bb82db229ed4ad441 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:33 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:57486
I20260807 20:08:33.566412 11061 tablet_replica.cc:330] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: stopping tablet replica
I20260807 20:08:33.566519 11061 raft_consensus.cc:2229] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:33.566622 11061 raft_consensus.cc:2258] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:33.566680 11060 ts_tablet_manager.cc:1905] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:33.567068 11061 ts_tablet_manager.cc:1905] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:33.567162 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 9642
I20260807 20:08:33.569567 11061 ts_tablet_manager.cc:1918] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:33.569648 11061 log.cc:1196] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/1f49654dd556416bb82db229ed4ad441
I20260807 20:08:33.569976 11061 ts_tablet_manager.cc:1939] T 1f49654dd556416bb82db229ed4ad441 P e286b497d371491ba4c552c0f9eb285b: Deleting consensus metadata
I20260807 20:08:33.570468 11062 tablet_replica.cc:330] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: stopping tablet replica
I20260807 20:08:33.570612 11062 raft_consensus.cc:2229] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:33.570708 11062 raft_consensus.cc:2258] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:33.571156 11062 ts_tablet_manager.cc:1905] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:33.571369 11060 ts_tablet_manager.cc:1918] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:33.571448 11060 log.cc:1196] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/1f49654dd556416bb82db229ed4ad441
I20260807 20:08:33.571779 11060 ts_tablet_manager.cc:1939] T 1f49654dd556416bb82db229ed4ad441 P 52da89c27ef146a19f5355741440587b: Deleting consensus metadata
I20260807 20:08:33.573267 11062 ts_tablet_manager.cc:1918] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:33.573334 11062 log.cc:1196] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/1f49654dd556416bb82db229ed4ad441
I20260807 20:08:33.573621 11062 ts_tablet_manager.cc:1939] T 1f49654dd556416bb82db229ed4ad441 P eac1c7997cf5407b91acb8d2c8d15500: Deleting consensus metadata
W20260807 20:08:33.576243 10739 connection.cc:535] server connection from 127.0.0.1:52154 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 20:08:33.576285 10597 connection.cc:535] server connection from 127.0.0.1:57486 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 20:08:33.576962 10882 connection.cc:535] server connection from 127.0.0.1:52850 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:08:33.585718 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.577984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44354 (local address 127.3.15.189:36985)
0807 20:08:33.578024 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.578026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.578113 (+ 87us) server_negotiation.cc:407] Connection header received
0807 20:08:33.578216 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.578217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.578230 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.578262 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.579120 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.579615 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.580582 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.580913 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.580953 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.582859 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.582860 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.582860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.582868 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.584424 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.584807 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.584809 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.584809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.584867 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.585075 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.585076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.585076 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.585215 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.585310 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.585526 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.585619 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:33.586108 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:33.577951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48388 (local address 127.3.15.188:44985)
0807 20:08:33.577992 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:33.577994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:33.578109 (+ 115us) server_negotiation.cc:407] Connection header received
0807 20:08:33.578189 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:33.578190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:33.578202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:33.578262 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:33.579093 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.579871 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.580827 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:33.581165 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:33.581237 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:33.582949 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:33.582950 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:33.582951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:33.582959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:33.584719 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.585145 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.585146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.585147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.585199 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:33.585404 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:33.585405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:33.585405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:33.585538 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:33.585631 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:33.585948 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 20:08:33.586030 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:34.562237 11070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:34.561946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:39527)
0807 20:08:34.562071 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:34.562182 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
I20260807 20:08:34.562228 11071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:34.561943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:34491)
0807 20:08:34.562089 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:34.562173 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
W20260807 20:08:34.562520 11001 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:33055 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
W20260807 20:08:34.562668 10857 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:33055 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:34.562816 11072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:34.562594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:49593)
0807 20:08:34.562701 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:34.562767 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
W20260807 20:08:34.563042 10716 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:33055 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:35.065768 11073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 23cafcb5ec1a429e9291a39af4c9e767)
I20260807 20:08:35.065915 11073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } }
I20260807 20:08:35.066213 11073 leader_election.cc:288] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055), 68ad022a655a483f9f322e8bb7dfb0c5 (127.3.15.189:36985)
I20260807 20:08:35.066515 11075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 23cafcb5ec1a429e9291a39af4c9e767)
I20260807 20:08:35.066660 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.066372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54244)
0807 20:08:35.066513 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.066572 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260807 20:08:35.066748 11075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } }
W20260807 20:08:35.066817 10507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:35.067253 11075 leader_election.cc:288] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055), 4df1e015721641e3902f224ee4cd99f5 (127.3.15.188:44985)
I20260807 20:08:35.067355 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.067266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54266)
0807 20:08:35.067303 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.067328 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:08:35.067423 10507 leader_election.cc:334] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055): Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:35.067467 11079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.067111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54250)
0807 20:08:35.067358 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.067408 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":60,"threads_started":1}
W20260807 20:08:35.067577 10308 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:35.068020 11080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.067914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54274)
0807 20:08:35.067951 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.067983 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260807 20:08:35.068110 10308 leader_election.cc:334] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055): Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:35.069841 11077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.066470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44368 (local address 127.3.15.189:36985)
0807 20:08:35.066582 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.066585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.066618 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:08:35.066720 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.066721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.066734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.066810 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.067051 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.067835 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.069054 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.069450 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.069535 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.069548 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.069730 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.069779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":54,"threads_started":1}
I20260807 20:08:35.069849 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.066432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36985 (local address 127.0.0.1:44368)
0807 20:08:35.066542 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.066554 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.066637 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.066874 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.066875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.066877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.066940 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.066943 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.067905 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.067906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.068969 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.068972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.069536 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.069537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.069607 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.069654 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.069712 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.069761 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":37,"thread_start_us":51,"threads_started":1}
I20260807 20:08:35.070052 10332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4df1e015721641e3902f224ee4cd99f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" is_pre_election: true
I20260807 20:08:35.070165 10332 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4df1e015721641e3902f224ee4cd99f5 in term 1.
I20260807 20:08:35.070341 10506 leader_election.cc:302] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4df1e015721641e3902f224ee4cd99f5, 68ad022a655a483f9f322e8bb7dfb0c5; no voters: 23cafcb5ec1a429e9291a39af4c9e767
I20260807 20:08:35.070448 11073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 20:08:35.070501 11073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 23cafcb5ec1a429e9291a39af4c9e767)
I20260807 20:08:35.070523 11073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:08:35.070740 11081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.067270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48392 (local address 127.3.15.188:44985)
0807 20:08:35.067724 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.067726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.067733 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:35.067755 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.067756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.067765 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.067840 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.067976 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.068544 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.069525 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.070432 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.070497 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.070507 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.070654 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.070685 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":430,"thread_start_us":81,"threads_started":1}
I20260807 20:08:35.070757 11079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.067170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44985 (local address 127.0.0.1:48392)
0807 20:08:35.067534 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.067543 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.067593 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.067846 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.067847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.067850 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.067892 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.067893 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.068641 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.068642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.069437 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.069439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.070512 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.070513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.070582 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.070587 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.070642 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.070689 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":48,"threads_started":1}
I20260807 20:08:35.070945 10531 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4df1e015721641e3902f224ee4cd99f5" is_pre_election: true
I20260807 20:08:35.071465 11073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } }
I20260807 20:08:35.071604 10531 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 68ad022a655a483f9f322e8bb7dfb0c5 in current term 2: Already voted for candidate 4df1e015721641e3902f224ee4cd99f5 in this term.
I20260807 20:08:35.071625 11073 leader_election.cc:288] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [CANDIDATE]: Term 2 election: Requested vote from peers 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055), 68ad022a655a483f9f322e8bb7dfb0c5 (127.3.15.189:36985)
I20260807 20:08:35.071780 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.071694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54288)
0807 20:08:35.071726 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.071757 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 20:08:35.071802 10305 leader_election.cc:302] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 68ad022a655a483f9f322e8bb7dfb0c5; no voters: 23cafcb5ec1a429e9291a39af4c9e767, 4df1e015721641e3902f224ee4cd99f5
I20260807 20:08:35.071844 10332 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4df1e015721641e3902f224ee4cd99f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "68ad022a655a483f9f322e8bb7dfb0c5"
I20260807 20:08:35.071898 10332 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:08:35.072510 10332 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4df1e015721641e3902f224ee4cd99f5 in term 2.
I20260807 20:08:35.072588 11075 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:08:35.072681 10506 leader_election.cc:302] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4df1e015721641e3902f224ee4cd99f5, 68ad022a655a483f9f322e8bb7dfb0c5; no voters:
I20260807 20:08:35.072762 11073 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:08:35.072880 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.072800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54304)
0807 20:08:35.072837 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.072859 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 20:08:35.072927 11073 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [term 2 LEADER]: Becoming Leader. State: Replica: 4df1e015721641e3902f224ee4cd99f5, State: Running, Role: LEADER
W20260807 20:08:35.072959 10507 leader_election.cc:334] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055): Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
I20260807 20:08:35.073027 11073 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } }
I20260807 20:08:35.073485 11083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4df1e015721641e3902f224ee4cd99f5. Latest consensus state: current_term: 2 leader_uuid: "4df1e015721641e3902f224ee4cd99f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:35.073583 11083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:35.073789 11085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:35.074295 11085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:35.074397 11085 catalog_manager.cc:1259] Loaded cluster ID: 162a9861a0894381897d0107e34039b0
I20260807 20:08:35.074432 11085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:35.074594 11085 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:35.074662 11085 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Loaded TSK: 0
I20260807 20:08:35.074826 11085 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:35.125442 10520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:35.127053 10520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:35.128181 10332 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [term 2 FOLLOWER]: Refusing update from remote peer 4df1e015721641e3902f224ee4cd99f5: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 20:08:35.128199 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.128030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54332)
0807 20:08:35.128086 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.128158 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:08:35.128460 11083 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 20:08:35.128618 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.128507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.0.0.1:54338)
0807 20:08:35.128560 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.128589 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 20:08:35.128762 10507 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 -> Peer 23cafcb5ec1a429e9291a39af4c9e767 (127.3.15.190:33055): Couldn't send request to peer 23cafcb5ec1a429e9291a39af4c9e767. Status: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 20:08:35.129122 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4df1e015721641e3902f224ee4cd99f5. Latest consensus state: current_term: 2 leader_uuid: "4df1e015721641e3902f224ee4cd99f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:35.129236 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:08:35.129712 11083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "4df1e015721641e3902f224ee4cd99f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:35.129827 11083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:35.130139 11075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "4df1e015721641e3902f224ee4cd99f5" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "23cafcb5ec1a429e9291a39af4c9e767" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33055 } } peers { permanent_uuid: "68ad022a655a483f9f322e8bb7dfb0c5" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36985 } attrs { promote: false } } peers { permanent_uuid: "4df1e015721641e3902f224ee4cd99f5" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44985 } attrs { promote: false } } }
I20260807 20:08:35.130250 11075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68ad022a655a483f9f322e8bb7dfb0c5 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:08:35.130343 11087 mvcc.cc:205] Tried to move back new op lower bound from 7316002058762895360 to 7316002058539794432. Current Snapshot: MvccSnapshot[applied={T|T < 7316002058762895360 or (T in {7316002058762895360})}]
I20260807 20:08:35.135048 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.131538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:45133 (local address 127.0.0.1:52162)
0807 20:08:35.131568 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.131578 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.131621 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.131899 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.131900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.131902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.131977 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.131979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.132905 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.132906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.134094 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.134095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.134793 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.134854 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.134859 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.134908 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.134955 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":5}
I20260807 20:08:35.135062 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.132172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.3.15.129:40975)
0807 20:08:35.132391 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.132393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.132410 (+ 17us) server_negotiation.cc:407] Connection header received
0807 20:08:35.132435 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.132436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.132446 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.132514 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.132627 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.133248 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134182 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.134688 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134773 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.134787 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.134947 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.134992 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":62,"threads_started":1}
I20260807 20:08:35.135136 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.131634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52162 (local address 127.3.15.130:45133)
0807 20:08:35.131761 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.131763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.131774 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:35.131804 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.131805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.131818 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.131888 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.132044 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.132846 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134195 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.134711 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134795 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.134808 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.135017 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.135065 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260807 20:08:35.135497 10788 tablet_service.cc:1467] Processing CreateTablet for tablet f8b81636588146fb9e654efd933842ed (DEFAULT_TABLE table=test-table [id=b62c9a48bd234ac096eb28b67d2e4e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:35.135586 11093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.131969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40975 (local address 127.0.0.1:57502)
0807 20:08:35.132205 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.132215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.132257 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.132516 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.132517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.132519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.132560 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.132561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.133339 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.133340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.134087 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.134089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.134771 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.134842 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.134847 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.134933 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.134983 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":73,"threads_started":1}
I20260807 20:08:35.135659 10788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b81636588146fb9e654efd933842ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:35.135746 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.131609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42541 (local address 127.0.0.1:52858)
0807 20:08:35.131757 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.131768 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.131820 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.132031 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.132032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.132035 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.132067 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.132067 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.132992 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.132992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.134082 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.134085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.135500 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.135502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.135567 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.135571 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.135630 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.135685 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":5}
I20260807 20:08:35.135813 10644 tablet_service.cc:1467] Processing CreateTablet for tablet f8b81636588146fb9e654efd933842ed (DEFAULT_TABLE table=test-table [id=b62c9a48bd234ac096eb28b67d2e4e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:35.135942 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b81636588146fb9e654efd933842ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:35.136044 11092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.131673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52858 (local address 127.3.15.131:42541)
0807 20:08:35.131805 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.131807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.131815 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:35.131895 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.131920 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.131930 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.132131 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.132173 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.132928 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.134845 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.135431 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.135500 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.135512 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.135917 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.135969 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":87,"threads_started":1}
I20260807 20:08:35.137280 11095 tablet_bootstrap.cc:492] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Bootstrap starting.
I20260807 20:08:35.137391 10930 tablet_service.cc:1467] Processing CreateTablet for tablet f8b81636588146fb9e654efd933842ed (DEFAULT_TABLE table=test-table [id=b62c9a48bd234ac096eb28b67d2e4e1a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:35.137534 10930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8b81636588146fb9e654efd933842ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:35.137632 11096 tablet_bootstrap.cc:492] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: Bootstrap starting.
I20260807 20:08:35.137820 11095 tablet_bootstrap.cc:654] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:35.138008 11096 tablet_bootstrap.cc:654] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:35.138598 11096 tablet_bootstrap.cc:492] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: No bootstrap required, opened a new log
I20260807 20:08:35.138655 11096 ts_tablet_manager.cc:1397] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:35.138756 11096 raft_consensus.cc:348] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.138839 11096 raft_consensus.cc:374] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:35.138859 11096 raft_consensus.cc:729] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac1c7997cf5407b91acb8d2c8d15500, State: Initialized, Role: FOLLOWER
I20260807 20:08:35.138932 11096 consensus_queue.cc:260] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.139087 11096 ts_tablet_manager.cc:1428] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:35.139107 11100 tablet_bootstrap.cc:492] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Bootstrap starting.
I20260807 20:08:35.139559 11100 tablet_bootstrap.cc:654] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:35.139786 10715 heartbeater.cc:503] Master 127.3.15.188:44985 was elected leader, sending a full tablet report...
I20260807 20:08:35.139858 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.139290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:59537)
0807 20:08:35.139780 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.139820 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":77,"threads_started":1}
I20260807 20:08:35.140239 11095 tablet_bootstrap.cc:492] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: No bootstrap required, opened a new log
I20260807 20:08:35.140249 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.140074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:44805)
0807 20:08:35.140198 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.140223 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260807 20:08:35.140309 11095 ts_tablet_manager.cc:1397] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 20:08:35.140421 11095 raft_consensus.cc:348] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.140506 11095 raft_consensus.cc:374] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:35.140535 11095 raft_consensus.cc:729] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e286b497d371491ba4c552c0f9eb285b, State: Initialized, Role: FOLLOWER
I20260807 20:08:35.140581 11095 consensus_queue.cc:260] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.140672 11100 tablet_bootstrap.cc:492] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: No bootstrap required, opened a new log
I20260807 20:08:35.140729 11100 ts_tablet_manager.cc:1397] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:35.140748 11095 ts_tablet_manager.cc:1428] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:35.140861 11100 raft_consensus.cc:348] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.140939 11100 raft_consensus.cc:374] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:35.140962 11100 raft_consensus.cc:729] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52da89c27ef146a19f5355741440587b, State: Initialized, Role: FOLLOWER
I20260807 20:08:35.141161 10856 heartbeater.cc:503] Master 127.3.15.188:44985 was elected leader, sending a full tablet report...
I20260807 20:08:35.141021 11100 consensus_queue.cc:260] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.141410 11100 ts_tablet_manager.cc:1428] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:35.141608 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.141010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:52045)
0807 20:08:35.141520 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.141557 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":83,"threads_started":1}
I20260807 20:08:35.141790 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.141520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:56049)
0807 20:08:35.141713 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.141746 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":70,"threads_started":1}
I20260807 20:08:35.141945 11000 heartbeater.cc:503] Master 127.3.15.188:44985 was elected leader, sending a full tablet report...
I20260807 20:08:35.141980 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.141810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:60115)
0807 20:08:35.141923 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.141948 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260807 20:08:35.142218 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.142038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:55637)
0807 20:08:35.142145 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.142178 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":69,"threads_started":1}
W20260807 20:08:35.174618 10718 tablet.cc:2369] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:35.198004 11106 raft_consensus.cc:482] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:35.198101 11106 raft_consensus.cc:504] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.198516 11106 leader_election.cc:288] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52da89c27ef146a19f5355741440587b (127.3.15.131:42541), eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975)
I20260807 20:08:35.202047 11107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.198663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:42541 (local address 127.3.15.130:59901)
0807 20:08:35.198774 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.198789 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.198870 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.199052 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.199053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.199055 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.199141 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.199143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.199988 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.199990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.201092 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.201095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.201715 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.201716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.201813 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.201825 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.201903 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.201965 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 20:08:35.202104 11092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.198755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:59901 (local address 127.3.15.131:42541)
0807 20:08:35.198805 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.198808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.198849 (+ 41us) server_negotiation.cc:407] Connection header received
0807 20:08:35.198964 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.198965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.198980 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.199037 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.199232 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.199896 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.201184 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.201588 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.201651 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.201660 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.201970 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.202025 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:08:35.202365 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.198776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:34187 (local address 127.3.15.129:40975)
0807 20:08:35.198832 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.198835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.198921 (+ 86us) server_negotiation.cc:407] Connection header received
0807 20:08:35.199040 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.199041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.199054 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.199102 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:35.199305 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.199778 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.201183 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.201823 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.201914 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.201929 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.202228 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.202289 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:08:35.202361 10952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b81636588146fb9e654efd933842ed" candidate_uuid: "e286b497d371491ba4c552c0f9eb285b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52da89c27ef146a19f5355741440587b" is_pre_election: true
I20260807 20:08:35.202486 10952 raft_consensus.cc:2454] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e286b497d371491ba4c552c0f9eb285b in term 0.
I20260807 20:08:35.202689 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.198700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40975 (local address 127.3.15.130:34187)
0807 20:08:35.198834 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.198845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:35.198940 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:35.199122 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:35.199123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:35.199125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:35.199230 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.199232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.199952 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.199953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.201090 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:35.201092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.202000 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.202010 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:35.202091 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.202101 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 20:08:35.202525 (+ 424us) client_negotiation.cc:240] Negotiation successful
0807 20:08:35.202579 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260807 20:08:35.202687 10741 leader_election.cc:302] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52da89c27ef146a19f5355741440587b, e286b497d371491ba4c552c0f9eb285b; no voters:
I20260807 20:08:35.202878 11106 raft_consensus.cc:2781] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:35.202929 11106 raft_consensus.cc:482] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:35.202955 11106 raft_consensus.cc:3037] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:35.203198 10666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b81636588146fb9e654efd933842ed" candidate_uuid: "e286b497d371491ba4c552c0f9eb285b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac1c7997cf5407b91acb8d2c8d15500" is_pre_election: true
I20260807 20:08:35.203330 10666 raft_consensus.cc:2454] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e286b497d371491ba4c552c0f9eb285b in term 0.
I20260807 20:08:35.203864 11106 raft_consensus.cc:504] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.204054 11106 leader_election.cc:288] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [CANDIDATE]: Term 1 election: Requested vote from peers 52da89c27ef146a19f5355741440587b (127.3.15.131:42541), eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975)
I20260807 20:08:35.204243 10666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b81636588146fb9e654efd933842ed" candidate_uuid: "e286b497d371491ba4c552c0f9eb285b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eac1c7997cf5407b91acb8d2c8d15500"
I20260807 20:08:35.204247 10952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8b81636588146fb9e654efd933842ed" candidate_uuid: "e286b497d371491ba4c552c0f9eb285b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52da89c27ef146a19f5355741440587b"
I20260807 20:08:35.204329 10952 raft_consensus.cc:3037] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:35.204329 10666 raft_consensus.cc:3037] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:35.205190 10952 raft_consensus.cc:2454] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e286b497d371491ba4c552c0f9eb285b in term 1.
I20260807 20:08:35.205297 10666 raft_consensus.cc:2454] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e286b497d371491ba4c552c0f9eb285b in term 1.
I20260807 20:08:35.205530 10740 leader_election.cc:302] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e286b497d371491ba4c552c0f9eb285b, eac1c7997cf5407b91acb8d2c8d15500; no voters:
I20260807 20:08:35.205646 11106 raft_consensus.cc:2781] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:35.205891 11106 raft_consensus.cc:686] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 1 LEADER]: Becoming Leader. State: Replica: e286b497d371491ba4c552c0f9eb285b, State: Running, Role: LEADER
I20260807 20:08:35.205991 11106 consensus_queue.cc:237] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } }
I20260807 20:08:35.206286 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.206150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:35927)
0807 20:08:35.206211 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.206254 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 20:08:35.206840 10520 catalog_manager.cc:5495] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b reported cstate change: term changed from 0 to 1, leader changed from <none> to e286b497d371491ba4c552c0f9eb285b (127.3.15.130). New cstate: current_term: 1 leader_uuid: "e286b497d371491ba4c552c0f9eb285b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e286b497d371491ba4c552c0f9eb285b" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 45133 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 } health_report { overall_health: UNKNOWN } } }
I20260807 20:08:35.227777 11091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.225497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52176 (local address 127.3.15.130:45133)
0807 20:08:35.225557 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:35.225559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:35.225582 (+ 23us) server_negotiation.cc:407] Connection header received
0807 20:08:35.225631 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:35.225632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:35.225652 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:35.225710 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:35.225843 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.226303 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.227031 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:35.227217 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:35.227267 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:35.227435 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:35.227540 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:35.227602 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:35.227675 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 20:08:35.227701 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 20:08:35.227783 11004 tablet.cc:2369] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:35.229048 10952 raft_consensus.cc:1261] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Refusing update from remote peer e286b497d371491ba4c552c0f9eb285b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:35.229199 10666 raft_consensus.cc:1261] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Refusing update from remote peer e286b497d371491ba4c552c0f9eb285b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:35.229399 11106 consensus_queue.cc:1035] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [LEADER]: Connected to new peer: Peer: permanent_uuid: "52da89c27ef146a19f5355741440587b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 42541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:35.229539 11113 consensus_queue.cc:1035] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eac1c7997cf5407b91acb8d2c8d15500" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:35.230511 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.230379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:39335)
0807 20:08:35.230430 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.230476 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:08:35.230810 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.230643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:38887)
0807 20:08:35.230743 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.230774 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 20:08:35.231951 11118 mvcc.cc:205] Tried to move back new op lower bound from 7316002059175829504 to 7316002059084455936. Current Snapshot: MvccSnapshot[applied={T|T < 7316002059175829504}]
I20260807 20:08:35.232285 11112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.232187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.131:35243)
0807 20:08:35.232224 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.232250 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 20:08:35.232501 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.232368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.130:33831)
0807 20:08:35.232439 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.232468 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 20:08:35.232530 11120 mvcc.cc:205] Tried to move back new op lower bound from 7316002059175829504 to 7316002059084455936. Current Snapshot: MvccSnapshot[applied={T|T < 7316002059175829504 or (T in {7316002059175829504})}]
I20260807 20:08:35.233266 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:35.233160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33055 (local address 127.3.15.129:39491)
0807 20:08:35.233198 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.233231 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:33055: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260807 20:08:35.234827 10520 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:35.234928 10520 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:35.236280 10520 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4df1e015721641e3902f224ee4cd99f5: Sending DeleteTablet for 3 replicas of tablet f8b81636588146fb9e654efd933842ed
I20260807 20:08:35.236552 10788 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b81636588146fb9e654efd933842ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:35 UTC) from {username='oryx', principal='oryx/127.3.15.188@KRBTEST.COM'} at 127.0.0.1:52162
I20260807 20:08:35.236590 10930 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b81636588146fb9e654efd933842ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:35 UTC) from {username='oryx', principal='oryx/127.3.15.188@KRBTEST.COM'} at 127.0.0.1:52858
I20260807 20:08:35.236804 10644 tablet_service.cc:1514] Processing DeleteTablet for tablet f8b81636588146fb9e654efd933842ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:35 UTC) from {username='oryx', principal='oryx/127.3.15.188@KRBTEST.COM'} at 127.0.0.1:57502
I20260807 20:08:35.236892 11128 tablet_replica.cc:330] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: stopping tablet replica
I20260807 20:08:35.236971 11128 raft_consensus.cc:2229] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:35.236989 11129 tablet_replica.cc:330] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500: stopping tablet replica
I20260807 20:08:35.237036 11129 raft_consensus.cc:2229] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:35.237051 11128 raft_consensus.cc:2258] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:35.237058 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 10575
I20260807 20:08:35.237124 11129 raft_consensus.cc:2258] T f8b81636588146fb9e654efd933842ed P eac1c7997cf5407b91acb8d2c8d15500 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:35.238129 11127 tablet_replica.cc:330] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: stopping tablet replica
I20260807 20:08:35.238197 11127 raft_consensus.cc:2229] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:35.238288 11127 raft_consensus.cc:2258] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:35.238566 11127 ts_tablet_manager.cc:1905] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:35.240113 11127 ts_tablet_manager.cc:1918] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:35.240190 11127 log.cc:1196] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/f8b81636588146fb9e654efd933842ed
I20260807 20:08:35.240408 11127 ts_tablet_manager.cc:1939] T f8b81636588146fb9e654efd933842ed P e286b497d371491ba4c552c0f9eb285b: Deleting consensus metadata
I20260807 20:08:35.240726 10505 catalog_manager.cc:4841] TS e286b497d371491ba4c552c0f9eb285b (127.3.15.130:45133): tablet f8b81636588146fb9e654efd933842ed (table test-table [id=b62c9a48bd234ac096eb28b67d2e4e1a]) successfully deleted
I20260807 20:08:35.245051 11128 ts_tablet_manager.cc:1905] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:35.246443 11128 ts_tablet_manager.cc:1918] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:35.246521 11128 log.cc:1196] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/f8b81636588146fb9e654efd933842ed
I20260807 20:08:35.246819 11128 ts_tablet_manager.cc:1939] T f8b81636588146fb9e654efd933842ed P 52da89c27ef146a19f5355741440587b: Deleting consensus metadata
I20260807 20:08:35.247170 10507 catalog_manager.cc:4841] TS 52da89c27ef146a19f5355741440587b (127.3.15.131:42541): tablet f8b81636588146fb9e654efd933842ed (table test-table [id=b62c9a48bd234ac096eb28b67d2e4e1a]) successfully deleted
W20260807 20:08:35.249668 11093 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:35.249534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40975 (local address 127.0.0.1:57508)
0807 20:08:35.249579 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:35.249608 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:40975: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 20:08:35.249810 10506 catalog_manager.cc:4584] TS eac1c7997cf5407b91acb8d2c8d15500 (127.3.15.129:40975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8b81636588146fb9e654efd933842ed: Network error: Client connection negotiation failed: client connection to 127.3.15.129:40975: connect: Connection reset by peer (error 104)
I20260807 20:08:35.249919 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 10722
I20260807 20:08:35.258235 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 10864
I20260807 20:08:35.264058 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 10182
I20260807 20:08:35.269848 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 10382
2026-08-07T20:08:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4759 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:35 dist-test-slave-1x4g krb5kdc[11132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:35 dist-test-slave-1x4g krb5kdc[11132](info): set up 2 sockets
Aug 07 20:08:35 dist-test-slave-1x4g krb5kdc[11132](info): commencing operation
krb5kdc: starting...
W20260807 20:08:36.338352 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab.
Entry for principal oryx/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab.
I20260807 20:08:36.461416 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39345
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab
--principal=oryx/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:36.547672 11145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:36.547865 11145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:36.547887 11145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:36.548954 11145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:36.548985 11145 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 20:08:36.548998 11145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:36.549011 11145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:36.549024 11145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:36.549037 11145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:36.550460 11145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39345
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.190.keytab
--principal=oryx/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:36.550633 11145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:36.550814 11145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:36.555600 11154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:36.555670 11145 server_base.cc:1034] running on GCE node
W20260807 20:08:36.555563 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:36.555564 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:36.556051 11145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:36.556365 11145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:36.557507 11145 hybrid_clock.cc:648] HybridClock initialized: now 1786133316557494 us; error 26 us; skew 500 ppm
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:36.560025 11145 init.cc:377] Logged in from keytab as oryx/127.3.15.190@KRBTEST.COM (short username oryx)
I20260807 20:08:36.560485 11145 webserver.cc:460] Webserver started at http://127.3.15.190:36007/ using document root <none> and password file <none>
I20260807 20:08:36.560719 11145 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:36.560765 11145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:36.560997 11145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:36.562029 11145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "a76e8900b5c14a45af474eb0001a7ab6"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "048ec2d193195a2b9f8cc31d7e457ac8"
server_key_iv: "cba9ce44009580dd294db7aeff16513b"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.562397 11145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "a76e8900b5c14a45af474eb0001a7ab6"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "048ec2d193195a2b9f8cc31d7e457ac8"
server_key_iv: "cba9ce44009580dd294db7aeff16513b"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.563699 11145 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:36.564459 11161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:36.564644 11145 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:36.564745 11145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "a76e8900b5c14a45af474eb0001a7ab6"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "048ec2d193195a2b9f8cc31d7e457ac8"
server_key_iv: "cba9ce44009580dd294db7aeff16513b"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.564853 11145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:36.575909 11145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:36.578311 11145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:36.578464 11145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:36.583324 11145 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39345
I20260807 20:08:36.583374 11226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39345 every 8 connection(s)
I20260807 20:08:36.583709 11145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:36.584343 11227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:36.586352 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 11145
I20260807 20:08:36.586476 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:36.586689 3134 external_mini_cluster.cc:1428] Setting key 2ea4e8fbb9337001b5a6e937546f50e2
I20260807 20:08:36.586683 11227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Bootstrap starting.
I20260807 20:08:36.587488 11227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:36.587832 11227 log.cc:824] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:36.588579 11227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: No bootstrap required, opened a new log
I20260807 20:08:36.591001 11227 raft_consensus.cc:348] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e8900b5c14a45af474eb0001a7ab6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39345 } }
I20260807 20:08:36.591148 11227 raft_consensus.cc:374] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:36.591171 11227 raft_consensus.cc:729] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a76e8900b5c14a45af474eb0001a7ab6, State: Initialized, Role: FOLLOWER
I20260807 20:08:36.591295 11227 consensus_queue.cc:260] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e8900b5c14a45af474eb0001a7ab6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39345 } }
I20260807 20:08:36.591380 11227 raft_consensus.cc:388] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:36.591419 11227 raft_consensus.cc:482] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:36.591466 11227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:36.592130 11227 raft_consensus.cc:504] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e8900b5c14a45af474eb0001a7ab6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39345 } }
I20260807 20:08:36.592248 11227 leader_election.cc:302] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a76e8900b5c14a45af474eb0001a7ab6; no voters:
I20260807 20:08:36.592435 11227 leader_election.cc:288] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:36.592517 11233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:36.592690 11233 raft_consensus.cc:686] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [term 1 LEADER]: Becoming Leader. State: Replica: a76e8900b5c14a45af474eb0001a7ab6, State: Running, Role: LEADER
I20260807 20:08:36.592736 11227 sys_catalog.cc:564] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:36.592861 11233 consensus_queue.cc:237] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e8900b5c14a45af474eb0001a7ab6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39345 } }
I20260807 20:08:36.593206 11233 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a76e8900b5c14a45af474eb0001a7ab6. Latest consensus state: current_term: 1 leader_uuid: "a76e8900b5c14a45af474eb0001a7ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e8900b5c14a45af474eb0001a7ab6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39345 } } }
I20260807 20:08:36.593294 11233 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:36.593226 11234 sys_catalog.cc:455] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a76e8900b5c14a45af474eb0001a7ab6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a76e8900b5c14a45af474eb0001a7ab6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39345 } } }
I20260807 20:08:36.593509 11234 sys_catalog.cc:458] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6 [sys.catalog]: This master's current role is: LEADER
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
W20260807 20:08:36.594720 11248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:08:36.594785 11248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:08:36.594863 11241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:36.595064 11241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:36.596635 11241 catalog_manager.cc:1347] Generated new cluster ID: 26ff31b96f1a42bea059e25c00a095ff
I20260807 20:08:36.596689 11241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:36.599354 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.587237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.3.15.190:39345)
0807 20:08:36.587438 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:36.587447 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:36.587484 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:08:36.588063 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:36.588072 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:36.588137 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:36.588253 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:36.589001 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.590445 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.591328 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.591945 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.592066 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.595090 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:36.595102 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:36.595103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:36.595116 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:36.597315 (+ 2199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.597723 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.597726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.597726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.597836 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.598065 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.598065 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.598066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.598379 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:36.598539 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:36.599000 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 20:08:36.599136 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260807 20:08:36.617834 11241 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:36.618609 11241 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:36.627441 11241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Generated new TSK 0
I20260807 20:08:36.627643 11241 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab.
Entry for principal oryx/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab.
I20260807 20:08:36.663556 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39345
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:36.747231 11254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:36.747452 11254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:36.747491 11254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:36.748624 11254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:36.748687 11254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:36.748721 11254 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:36.748772 11254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:36.750308 11254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:39345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:36.750573 11254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:36.750805 11254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:08:36.755944 11254 server_base.cc:1034] running on GCE node
W20260807 20:08:36.755864 11260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:36.755851 11259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:36.755853 11262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:36.756350 11254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:36.756552 11254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:36.757732 11254 hybrid_clock.cc:648] HybridClock initialized: now 1786133316757715 us; error 48 us; skew 500 ppm
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:36.760277 11254 init.cc:377] Logged in from keytab as oryx/127.3.15.129@KRBTEST.COM (short username oryx)
I20260807 20:08:36.760700 11254 webserver.cc:460] Webserver started at http://127.3.15.129:45429/ using document root <none> and password file <none>
I20260807 20:08:36.760951 11254 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:36.761029 11254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:36.761180 11254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:36.762213 11254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "8648c8cc46f2418eb76b429fb2419598"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "8d076ee845a0e043c01233168887ec3b"
server_key_iv: "0306aaf8a381e2ea23002ada227cc23a"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.762579 11254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "8648c8cc46f2418eb76b429fb2419598"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "8d076ee845a0e043c01233168887ec3b"
server_key_iv: "0306aaf8a381e2ea23002ada227cc23a"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.763899 11254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:36.764606 11270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:36.764822 11254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:36.764930 11254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "8648c8cc46f2418eb76b429fb2419598"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "8d076ee845a0e043c01233168887ec3b"
server_key_iv: "0306aaf8a381e2ea23002ada227cc23a"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.765041 11254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:36.796936 11254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:36.798837 11254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:36.799014 11254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:36.799583 11254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:36.799659 11254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:36.799703 11254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:36.799746 11254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:36.806782 11254 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40871
I20260807 20:08:36.806867 11389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40871 every 8 connection(s)
I20260807 20:08:36.807210 11254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:36.809804 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 11254
I20260807 20:08:36.809935 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:36.810006 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.800126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:34625 (local address 127.3.15.190:39345)
0807 20:08:36.800202 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:36.800205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:36.800381 (+ 176us) server_negotiation.cc:407] Connection header received
0807 20:08:36.800942 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:36.800943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:36.800957 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:36.801042 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:36.801888 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.802757 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.804378 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.804732 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.804861 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.807212 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:36.807213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:36.807214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:36.807222 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:36.808680 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.809093 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.809094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.809095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.809150 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.809373 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.809374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.809374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.809490 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:36.809541 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:36.809813 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 20:08:36.809913 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:08:36.810173 3134 external_mini_cluster.cc:1428] Setting key a72d44c26f8aca69ea38193ca2adc611
I20260807 20:08:36.810177 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.800080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.3.15.129:34625)
0807 20:08:36.800247 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:36.800289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:36.800832 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:36.801147 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:36.801150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:36.801465 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:36.801763 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.801769 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.802880 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.802881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.804268 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.804271 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.804891 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.804892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.804999 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.805543 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:36.805552 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:36.807116 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:36.808752 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.808758 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.808760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.809025 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.809214 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.809215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.809215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.809317 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.809544 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:36.809549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:36.809663 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 20:08:36.809769 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 20:08:36.809930 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260807 20:08:36.810691 11390 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39345
I20260807 20:08:36.810801 11390 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:36.810998 11390 heartbeater.cc:511] Master 127.3.15.190:39345 requested a full tablet report, sending...
I20260807 20:08:36.811620 11177 ts_manager.cc:194] Registered new tserver with Master: 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871)
I20260807 20:08:36.812516 11177 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:34625
I20260807 20:08:36.818645 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.812724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:36291 (local address 127.3.15.190:39345)
0807 20:08:36.812816 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:36.812818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:36.812827 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:36.812847 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:36.812848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:36.812859 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:36.812915 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:36.813353 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.813840 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.814867 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.815038 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.815086 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.816100 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:36.816101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:36.816101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:36.816107 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:36.817494 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.817904 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.817904 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.817905 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.817951 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.818183 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.818184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.818184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.818294 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:36.818347 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:36.818528 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 20:08:36.818586 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:08:36.818799 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.811840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.3.15.129:36291)
0807 20:08:36.811963 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:36.811974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:36.812025 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:36.812943 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:36.812944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:36.813173 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:36.813263 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.813265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.813942 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.813943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.814786 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.814789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.815109 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.815111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.815183 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.815576 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:36.815590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:36.816017 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:36.817573 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.817575 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.817575 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.817829 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.818017 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.818018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.818018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.818091 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.818362 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:36.818363 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:36.818415 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 20:08:36.818488 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:08:36.818618 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab.
Entry for principal oryx/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab.
I20260807 20:08:36.838963 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39345
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:36.922730 11399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:36.923040 11399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:36.923089 11399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:36.924261 11399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:36.924335 11399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:36.924369 11399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:36.924407 11399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:36.925982 11399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:39345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:36.926268 11399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:36.926501 11399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:36.931558 11405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:36.931568 11404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:36.931777 11399 server_base.cc:1034] running on GCE node
W20260807 20:08:36.931564 11407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:36.932054 11399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:36.932317 11399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:36.933485 11399 hybrid_clock.cc:648] HybridClock initialized: now 1786133316933448 us; error 39 us; skew 500 ppm
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:36.936049 11399 init.cc:377] Logged in from keytab as oryx/127.3.15.130@KRBTEST.COM (short username oryx)
I20260807 20:08:36.936481 11399 webserver.cc:460] Webserver started at http://127.3.15.130:37529/ using document root <none> and password file <none>
I20260807 20:08:36.936743 11399 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:36.936834 11399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:36.936985 11399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:36.938002 11399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "e3403af13a55471ba6a4fbf9954175f3"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "d5054b12cb6bf15ceaf9b0138d09cb00"
server_key_iv: "e11d1b0238a9ca319f85f59a67422e1b"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.938350 11399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "e3403af13a55471ba6a4fbf9954175f3"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "d5054b12cb6bf15ceaf9b0138d09cb00"
server_key_iv: "e11d1b0238a9ca319f85f59a67422e1b"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.939705 11399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:36.940428 11414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:36.940591 11399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:36.940696 11399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "e3403af13a55471ba6a4fbf9954175f3"
format_stamp: "Formatted at 2026-08-07 20:08:36 on dist-test-slave-1x4g"
server_key: "d5054b12cb6bf15ceaf9b0138d09cb00"
server_key_iv: "e11d1b0238a9ca319f85f59a67422e1b"
server_key_version: "encryptionkey@0"
I20260807 20:08:36.940766 11399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:36.950826 11399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:36.953224 11399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:36.953382 11399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:36.953939 11399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:36.953997 11399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:36.954039 11399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:36.954067 11399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 20:08:36 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:36.960966 11399 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:41893
I20260807 20:08:36.961038 11538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:41893 every 8 connection(s)
I20260807 20:08:36.961371 11399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:36.963869 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.954405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46375 (local address 127.3.15.190:39345)
0807 20:08:36.954479 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:36.954482 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:36.954632 (+ 150us) server_negotiation.cc:407] Connection header received
0807 20:08:36.955122 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:36.955124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:36.955137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:36.955219 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:36.956038 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.956945 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.958560 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.958905 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.958981 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.961320 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:36.961321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:36.961322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:36.961330 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:36.962651 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.963033 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.963035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.963035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.963103 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.963336 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.963337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.963337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.963457 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:36.963525 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:36.963727 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 20:08:36.963793 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:08:36.964022 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.954360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.3.15.130:46375)
0807 20:08:36.954506 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:36.954545 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:36.955023 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:36.955283 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:36.955287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:36.955642 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:36.955926 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.955932 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.957041 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.957043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.958480 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.958484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.958990 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.958991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.959130 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.959679 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:36.959688 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:36.961226 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:36.962718 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.962724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.962725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.962935 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.963176 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.963177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.963177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.963273 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.963515 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:36.963517 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:36.963633 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 20:08:36.963705 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 20:08:36.963819 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
I20260807 20:08:36.964200 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 11399
I20260807 20:08:36.964320 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:36.964570 3134 external_mini_cluster.cc:1428] Setting key ff2f6138e141db76c0d39a39a723e12a
I20260807 20:08:36.964818 11539 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39345
I20260807 20:08:36.964942 11539 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:36.965155 11539 heartbeater.cc:511] Master 127.3.15.190:39345 requested a full tablet report, sending...
I20260807 20:08:36.965556 11178 ts_manager.cc:194] Registered new tserver with Master: e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893)
I20260807 20:08:36.965976 11178 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:46375
I20260807 20:08:36.973234 11545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.965617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.3.15.130:49219)
0807 20:08:36.965758 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:36.965769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:36.965816 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:36.967433 (+ 1617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:36.967434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:36.967579 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:36.967668 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.967670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.968558 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.968559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.969135 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:36.969137 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.969600 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.969601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:36.969648 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.970115 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:36.970120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:36.970418 (+ 298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:36.972179 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.972180 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.972180 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.972343 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.972559 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:36.972559 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:36.972560 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:36.972612 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:36.972981 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:36.972981 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:36.973028 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:08:36.973082 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 20:08:36.973168 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260807 20:08:36.973340 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:36.967232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:49219 (local address 127.3.15.190:39345)
0807 20:08:36.967283 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:36.967285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:36.967295 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:36.967317 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:36.967318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:36.967328 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:36.967393 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:36.967731 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.968464 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.969216 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:36.969527 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:36.969600 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:36.970509 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:36.970511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:36.970511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:36.970519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:36.972102 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.972416 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.972417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.972417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.972483 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:36.972677 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:36.972678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:36.972678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:36.972896 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:36.972992 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:36.973132 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 20:08:36.973227 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for oryx/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab.
Entry for principal oryx/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab.
I20260807 20:08:36.991496 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39345
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:37.075533 11550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:37.075743 11550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:37.075766 11550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:37.077018 11550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:37.077059 11550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:37.077080 11550 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 20:08:37.077139 11550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:37.078691 11550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.15.190:39345
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:37.078938 11550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:37.079190 11550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:37.084138 11555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:37.084139 11558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:37.084326 11550 server_base.cc:1034] running on GCE node
W20260807 20:08:37.084139 11556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:37.084710 11550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:37.084997 11550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:37.086200 11550 hybrid_clock.cc:648] HybridClock initialized: now 1786133317086184 us; error 24 us; skew 500 ppm
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133317, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:37.089219 11550 init.cc:377] Logged in from keytab as oryx/127.3.15.131@KRBTEST.COM (short username oryx)
I20260807 20:08:37.089704 11550 webserver.cc:460] Webserver started at http://127.3.15.131:46605/ using document root <none> and password file <none>
I20260807 20:08:37.089942 11550 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:37.090013 11550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:37.090166 11550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:37.091197 11550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "840caf72a91a4bacb49f361f42494d2e"
format_stamp: "Formatted at 2026-08-07 20:08:37 on dist-test-slave-1x4g"
server_key: "6befb7a0c96118f0e2b33e248c1a7395"
server_key_iv: "17f7c2a941fe76bb15ad038d2af81070"
server_key_version: "encryptionkey@0"
I20260807 20:08:37.091572 11550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "840caf72a91a4bacb49f361f42494d2e"
format_stamp: "Formatted at 2026-08-07 20:08:37 on dist-test-slave-1x4g"
server_key: "6befb7a0c96118f0e2b33e248c1a7395"
server_key_iv: "17f7c2a941fe76bb15ad038d2af81070"
server_key_version: "encryptionkey@0"
I20260807 20:08:37.093091 11550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:08:37.094060 11565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.094420 11550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:37.094516 11550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "840caf72a91a4bacb49f361f42494d2e"
format_stamp: "Formatted at 2026-08-07 20:08:37 on dist-test-slave-1x4g"
server_key: "6befb7a0c96118f0e2b33e248c1a7395"
server_key_iv: "17f7c2a941fe76bb15ad038d2af81070"
server_key_version: "encryptionkey@0"
I20260807 20:08:37.094583 11550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:37.108392 11550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:37.110275 11550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:37.110383 11550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:37.110922 11550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:37.110984 11550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.111028 11550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:37.111056 11550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.117722 11550 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:46651
I20260807 20:08:37.117800 11688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:46651 every 8 connection(s)
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133317, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:37.118139 11550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:37.121322 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.111325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:52463 (local address 127.3.15.190:39345)
0807 20:08:37.111367 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.111370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.111591 (+ 221us) server_negotiation.cc:407] Connection header received
0807 20:08:37.112153 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.112154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.112168 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.112233 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.113054 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.113882 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.115470 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.115739 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.115827 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.118621 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:37.118623 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:37.118624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:37.118633 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:37.120178 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.120548 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.120549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.120549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.120604 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.120817 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.120818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.120818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.120928 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:37.120983 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.121195 (+ 212us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.121258 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:08:37.121498 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.111330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.3.15.131:52463)
0807 20:08:37.111480 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.111513 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.112047 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.112304 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.112307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.112624 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:37.112934 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.112942 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.114008 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.114009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.115389 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.115394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.115839 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.115840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.115989 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.116515 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:37.116526 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:37.118468 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:37.120252 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:37.120257 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:37.120259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:37.120481 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:37.120646 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:37.120646 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:37.120647 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:37.120735 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:37.120987 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:37.120992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:37.121101 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.121184 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.121300 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":94,"threads_started":1}
I20260807 20:08:37.121986 11689 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39345
I20260807 20:08:37.122071 11689 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:37.122248 11689 heartbeater.cc:511] Master 127.3.15.190:39345 requested a full tablet report, sending...
I20260807 20:08:37.122651 11177 ts_manager.cc:194] Registered new tserver with Master: 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.123140 11177 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:52463
I20260807 20:08:37.127313 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 11550
I20260807 20:08:37.127455 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:37.127699 3134 external_mini_cluster.cc:1428] Setting key 41c59d8ae34b32dac899140ea63059bf
I20260807 20:08:37.128523 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.123087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:47133 (local address 127.3.15.190:39345)
0807 20:08:37.123129 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.123130 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.123216 (+ 86us) server_negotiation.cc:407] Connection header received
0807 20:08:37.123314 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.123315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.123325 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.123351 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.123695 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.124218 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.124904 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.125095 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.125149 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.125962 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:37.125963 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:37.125963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:37.125968 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:37.127064 (+ 1096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.127633 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.127634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.127635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.127689 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.127942 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.127944 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.127944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.128071 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:37.128153 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.128371 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.128439 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:37.128536 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.123021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.3.15.131:47133)
0807 20:08:37.123154 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.123164 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.123242 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.123411 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.123411 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.123564 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:37.123621 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.123622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.124302 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.124303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.124833 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.124835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.125159 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.125160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.125207 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.125620 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:37.125625 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:37.125887 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:37.127145 (+ 1258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:37.127146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:37.127147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:37.127384 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:37.127765 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:37.127766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:37.127766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:37.127841 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:37.128210 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:37.128211 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:37.128259 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.128329 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.128433 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260807 20:08:37.128561 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 20:08:37.151625 11230 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:37.146168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.3.15.190:39345)
0807 20:08:37.146235 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.146237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.146266 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:08:37.146352 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.146352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.146361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.146403 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.146953 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.147437 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.148171 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.148349 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.148397 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.151504 (+ 3107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43540: BlockingRecv error: recv got EOF from 127.0.0.1:43540 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133317, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:37.159152 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.152338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43552 (local address 127.3.15.190:39345)
0807 20:08:37.152382 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.152384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.152465 (+ 81us) server_negotiation.cc:407] Connection header received
0807 20:08:37.152552 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.152553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.152563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.152625 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.153164 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.153634 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.154322 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.154506 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.154572 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.156710 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:37.156712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:37.156713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:37.156720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:37.158034 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.158370 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.158372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.158372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.158432 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.158637 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.158638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.158638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.158749 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:37.158851 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.158969 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.159046 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:08:37.160710 11177 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:37.162415 11177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:37.169430 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.165060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.3.15.131:46651)
0807 20:08:37.165495 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.165544 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.165556 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:08:37.165607 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.165609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.165624 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.165722 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.165914 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.166727 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.168258 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.168967 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.169005 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.169059 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.169288 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.169326 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":93,"threads_started":1}
I20260807 20:08:37.169413 11712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.164929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46651 (local address 127.0.0.1:55836)
0807 20:08:37.165098 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.165138 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.165212 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.165744 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.165748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.165751 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.165819 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.165820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.166807 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.166807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.168155 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.168158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.169109 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.169110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.169180 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.169189 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.169272 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.169326 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":56,"threads_started":1}
I20260807 20:08:37.169775 11714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.165138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40871 (local address 127.0.0.1:60110)
0807 20:08:37.165813 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.165823 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.165874 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.166083 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.166084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.166086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.166119 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.166121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.167356 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.167357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.168408 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.168424 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.169499 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.169500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.169574 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.169578 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.169651 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.169692 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":1,"thread_start_us":81,"threads_started":1}
I20260807 20:08:37.169834 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.165262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60110 (local address 127.3.15.129:40871)
0807 20:08:37.165465 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.165469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.165933 (+ 464us) server_negotiation.cc:407] Connection header received
0807 20:08:37.165977 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.165979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.165994 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.166071 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.166185 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.167274 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.168539 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.169311 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.169384 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.169460 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.169682 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.169726 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260807 20:08:37.169894 11711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.164944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:41893 (local address 127.0.0.1:41360)
0807 20:08:37.165457 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.165469 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.165524 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.165857 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.165858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.165860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.165906 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.165907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.167259 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.167260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.168403 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.168405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.169545 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.169545 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.169599 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.169602 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.169651 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.169692 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":98,"threads_started":1}
I20260807 20:08:37.170135 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.165259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41360 (local address 127.3.15.130:41893)
0807 20:08:37.165636 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.165640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.165650 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:37.165692 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.165695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.165712 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.165790 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.165982 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.167186 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.168477 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.169311 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.169384 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.169459 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.169737 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.169776 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":114,"threads_started":1}
I20260807 20:08:37.170621 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 9a92214564a54d45902de5d01e319685 (DEFAULT_TABLE table=test-table [id=a2a3b7b7a490403cb1e34d420599ce9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.170573 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 9a92214564a54d45902de5d01e319685 (DEFAULT_TABLE table=test-table [id=a2a3b7b7a490403cb1e34d420599ce9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.170964 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a92214564a54d45902de5d01e319685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.170969 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a92214564a54d45902de5d01e319685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.173925 11718 tablet_bootstrap.cc:492] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Bootstrap starting.
I20260807 20:08:37.174233 11719 tablet_bootstrap.cc:492] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Bootstrap starting.
I20260807 20:08:37.174634 11718 tablet_bootstrap.cc:654] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.174870 11719 tablet_bootstrap.cc:654] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.174741 11469 tablet_service.cc:1467] Processing CreateTablet for tablet 9a92214564a54d45902de5d01e319685 (DEFAULT_TABLE table=test-table [id=a2a3b7b7a490403cb1e34d420599ce9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.174979 11718 log.cc:824] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:37.175112 11469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a92214564a54d45902de5d01e319685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.175233 11719 log.cc:824] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:37.176064 11719 tablet_bootstrap.cc:492] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: No bootstrap required, opened a new log
I20260807 20:08:37.176136 11719 ts_tablet_manager.cc:1397] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:37.178355 11719 raft_consensus.cc:348] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.178542 11719 raft_consensus.cc:374] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.178583 11719 raft_consensus.cc:729] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.178723 11719 consensus_queue.cc:260] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.179018 11719 ts_tablet_manager.cc:1428] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:08:37.179183 11390 heartbeater.cc:503] Master 127.3.15.190:39345 was elected leader, sending a full tablet report...
I20260807 20:08:37.179296 11723 tablet_bootstrap.cc:492] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Bootstrap starting.
I20260807 20:08:37.179458 11718 tablet_bootstrap.cc:492] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: No bootstrap required, opened a new log
I20260807 20:08:37.179533 11718 ts_tablet_manager.cc:1397] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 20:08:37.179967 11723 tablet_bootstrap.cc:654] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.180341 11723 log.cc:824] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:37.181104 11723 tablet_bootstrap.cc:492] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: No bootstrap required, opened a new log
I20260807 20:08:37.181187 11723 ts_tablet_manager.cc:1397] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:08:37.181535 11718 raft_consensus.cc:348] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.181668 11718 raft_consensus.cc:374] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.181690 11718 raft_consensus.cc:729] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840caf72a91a4bacb49f361f42494d2e, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.181816 11718 consensus_queue.cc:260] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.182091 11718 ts_tablet_manager.cc:1428] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:08:37.182132 11689 heartbeater.cc:503] Master 127.3.15.190:39345 was elected leader, sending a full tablet report...
I20260807 20:08:37.183193 11723 raft_consensus.cc:348] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.183347 11723 raft_consensus.cc:374] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.183373 11723 raft_consensus.cc:729] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3403af13a55471ba6a4fbf9954175f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.183486 11723 consensus_queue.cc:260] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.183745 11723 ts_tablet_manager.cc:1428] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:08:37.183796 11539 heartbeater.cc:503] Master 127.3.15.190:39345 was elected leader, sending a full tablet report...
W20260807 20:08:37.212001 11540 tablet.cc:2369] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:37.266880 11722 raft_consensus.cc:482] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:37.267076 11722 raft_consensus.cc:504] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.267479 11722 leader_election.cc:288] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651), e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893)
I20260807 20:08:37.271428 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.267684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:41893 (local address 127.3.15.129:55013)
0807 20:08:37.267860 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.267869 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.267918 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.268083 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.268083 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.268084 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.268165 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.268167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.269075 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.269076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.270386 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.270389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.271122 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.271123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.271199 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.271207 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.271266 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.271315 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":104,"thread_start_us":75,"threads_started":1}
I20260807 20:08:37.271458 11713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.267767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:54461 (local address 127.3.15.131:46651)
0807 20:08:37.267813 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.267815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.267826 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:37.267862 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.267863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.267875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.267976 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.268208 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.268985 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.270481 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.271029 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.271104 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.271112 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.271333 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.271381 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:37.271453 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.267796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:55013 (local address 127.3.15.130:41893)
0807 20:08:37.267838 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.267841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.267931 (+ 90us) server_negotiation.cc:407] Connection header received
0807 20:08:37.267998 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.267999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.268012 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.268071 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.268233 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.268993 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.270481 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.271026 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.271104 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.271112 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.271333 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.271390 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:08:37.271438 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.267682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46651 (local address 127.3.15.129:54461)
0807 20:08:37.267732 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.267747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.267800 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.267992 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.267993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.267995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.268100 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.268103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.269075 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.269076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.270386 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.270389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.271112 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.271114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.271199 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.271206 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.271266 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.271315 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":36}
I20260807 20:08:37.271790 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a92214564a54d45902de5d01e319685" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e" is_pre_election: true
I20260807 20:08:37.271802 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a92214564a54d45902de5d01e319685" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3" is_pre_election: true
I20260807 20:08:37.271940 11642 raft_consensus.cc:2454] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.271966 11489 raft_consensus.cc:2454] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.272197 11274 leader_election.cc:302] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, 8648c8cc46f2418eb76b429fb2419598; no voters:
I20260807 20:08:37.272343 11722 raft_consensus.cc:2781] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:37.272426 11722 raft_consensus.cc:482] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:37.272460 11722 raft_consensus.cc:3037] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.273275 11722 raft_consensus.cc:504] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.273425 11722 leader_election.cc:288] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Requested vote from peers 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651), e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893)
I20260807 20:08:37.273584 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a92214564a54d45902de5d01e319685" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e"
I20260807 20:08:37.273590 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a92214564a54d45902de5d01e319685" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3"
I20260807 20:08:37.273682 11642 raft_consensus.cc:3037] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.273680 11489 raft_consensus.cc:3037] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.274389 11489 raft_consensus.cc:2454] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.274482 11642 raft_consensus.cc:2454] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.275005 11273 leader_election.cc:302] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8648c8cc46f2418eb76b429fb2419598, e3403af13a55471ba6a4fbf9954175f3; no voters:
I20260807 20:08:37.275118 11722 raft_consensus.cc:2781] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:37.275269 11722 raft_consensus.cc:686] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Becoming Leader. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Running, Role: LEADER
I20260807 20:08:37.275377 11722 consensus_queue.cc:237] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } }
I20260807 20:08:37.276252 11177 catalog_manager.cc:5495] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:37.287194 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.285060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.3.15.129:40871)
0807 20:08:37.285132 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.285134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.285142 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:37.285190 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.285191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.285208 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.285266 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.285383 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.285826 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.286529 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.286708 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.286758 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.286895 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.286988 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.287046 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.287080 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.287103 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 20:08:37.288959 11489 raft_consensus.cc:1261] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.288959 11642 raft_consensus.cc:1261] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.289309 11722 consensus_queue.cc:1035] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.289463 11728 consensus_queue.cc:1035] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.291806 11732 mvcc.cc:205] Tried to move back new op lower bound from 7316002067612962816 to 7316002067560837120. Current Snapshot: MvccSnapshot[applied={T|T < 7316002067612962816}]
I20260807 20:08:37.293076 11734 mvcc.cc:205] Tried to move back new op lower bound from 7316002067612962816 to 7316002067560837120. Current Snapshot: MvccSnapshot[applied={T|T < 7316002067612962816 or (T in {7316002067612962816})}]
I20260807 20:08:37.294126 11178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.294215 11178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.295075 11178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Sending DeleteTablet for 3 replicas of tablet 9a92214564a54d45902de5d01e319685
I20260807 20:08:37.295401 11618 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a92214564a54d45902de5d01e319685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:55836
I20260807 20:08:37.295658 11743 tablet_replica.cc:330] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: stopping tablet replica
I20260807 20:08:37.295642 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a92214564a54d45902de5d01e319685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:60110
I20260807 20:08:37.295794 11743 raft_consensus.cc:2229] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.295805 11469 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a92214564a54d45902de5d01e319685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41360
I20260807 20:08:37.296123 11178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 20:08:37.296263 11745 tablet_replica.cc:330] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: stopping tablet replica
W20260807 20:08:37.296301 11178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:37.296221 11743 raft_consensus.cc:2258] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.296355 11745 raft_consensus.cc:2229] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.297024 11743 ts_tablet_manager.cc:1905] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.296751 11744 tablet_replica.cc:330] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: stopping tablet replica
I20260807 20:08:37.298167 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 59cfd935f8434229942f6c76bc5e0f87 (DEFAULT_TABLE table=test-table [id=51d55f01ec20466c89422b09d2a39315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.298244 11469 tablet_service.cc:1467] Processing CreateTablet for tablet 59cfd935f8434229942f6c76bc5e0f87 (DEFAULT_TABLE table=test-table [id=51d55f01ec20466c89422b09d2a39315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.298305 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59cfd935f8434229942f6c76bc5e0f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.298291 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 59cfd935f8434229942f6c76bc5e0f87 (DEFAULT_TABLE table=test-table [id=51d55f01ec20466c89422b09d2a39315]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.298362 11469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59cfd935f8434229942f6c76bc5e0f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.298388 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59cfd935f8434229942f6c76bc5e0f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.299630 11719 tablet_bootstrap.cc:492] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Bootstrap starting.
I20260807 20:08:37.299656 11718 tablet_bootstrap.cc:492] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Bootstrap starting.
I20260807 20:08:37.299713 11723 tablet_bootstrap.cc:492] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Bootstrap starting.
I20260807 20:08:37.300062 11719 tablet_bootstrap.cc:654] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.300105 11718 tablet_bootstrap.cc:654] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.300143 11723 tablet_bootstrap.cc:654] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.300501 11745 raft_consensus.cc:2258] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.300876 11719 tablet_bootstrap.cc:492] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: No bootstrap required, opened a new log
I20260807 20:08:37.300930 11719 ts_tablet_manager.cc:1397] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:37.300985 11744 raft_consensus.cc:2229] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:37.301111 11744 raft_consensus.cc:2258] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.301082 11719 raft_consensus.cc:348] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.301177 11719 raft_consensus.cc:374] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.301208 11719 raft_consensus.cc:729] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.301250 11719 consensus_queue.cc:260] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.301357 11719 ts_tablet_manager.cc:1428] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.301573 11745 ts_tablet_manager.cc:1905] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.301716 11744 ts_tablet_manager.cc:1905] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.303170 11745 ts_tablet_manager.cc:1918] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:37.303251 11745 log.cc:1196] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/9a92214564a54d45902de5d01e319685
I20260807 20:08:37.303390 11744 ts_tablet_manager.cc:1918] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:37.303474 11744 log.cc:1196] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/9a92214564a54d45902de5d01e319685
I20260807 20:08:37.303539 11745 ts_tablet_manager.cc:1939] T 9a92214564a54d45902de5d01e319685 P e3403af13a55471ba6a4fbf9954175f3: Deleting consensus metadata
I20260807 20:08:37.303737 11744 ts_tablet_manager.cc:1939] T 9a92214564a54d45902de5d01e319685 P 8648c8cc46f2418eb76b429fb2419598: Deleting consensus metadata
I20260807 20:08:37.303980 11164 catalog_manager.cc:4841] TS e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893): tablet 9a92214564a54d45902de5d01e319685 (table test-table [id=a2a3b7b7a490403cb1e34d420599ce9c]) successfully deleted
I20260807 20:08:37.304100 11165 catalog_manager.cc:4841] TS 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871): tablet 9a92214564a54d45902de5d01e319685 (table test-table [id=a2a3b7b7a490403cb1e34d420599ce9c]) successfully deleted
I20260807 20:08:37.304177 11718 tablet_bootstrap.cc:492] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: No bootstrap required, opened a new log
I20260807 20:08:37.304225 11718 ts_tablet_manager.cc:1397] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260807 20:08:37.304364 11718 raft_consensus.cc:348] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.304461 11718 raft_consensus.cc:374] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.304489 11718 raft_consensus.cc:729] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840caf72a91a4bacb49f361f42494d2e, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.304543 11718 consensus_queue.cc:260] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.304616 11743 ts_tablet_manager.cc:1918] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:37.304661 11718 ts_tablet_manager.cc:1428] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:37.304816 11743 log.cc:1196] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/9a92214564a54d45902de5d01e319685
I20260807 20:08:37.305320 11743 ts_tablet_manager.cc:1939] T 9a92214564a54d45902de5d01e319685 P 840caf72a91a4bacb49f361f42494d2e: Deleting consensus metadata
I20260807 20:08:37.305706 11165 catalog_manager.cc:4841] TS 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651): tablet 9a92214564a54d45902de5d01e319685 (table test-table [id=a2a3b7b7a490403cb1e34d420599ce9c]) successfully deleted
I20260807 20:08:37.305838 11723 tablet_bootstrap.cc:492] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: No bootstrap required, opened a new log
I20260807 20:08:37.305892 11723 ts_tablet_manager.cc:1397] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260807 20:08:37.306036 11723 raft_consensus.cc:348] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.306113 11723 raft_consensus.cc:374] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.306134 11723 raft_consensus.cc:729] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3403af13a55471ba6a4fbf9954175f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.306182 11723 consensus_queue.cc:260] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.306267 11723 ts_tablet_manager.cc:1428] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260807 20:08:37.308239 11391 tablet.cc:2369] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:08:37.368705 11690 tablet.cc:2369] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:08:37.435669 11737 raft_consensus.cc:482] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:37.435786 11737 raft_consensus.cc:504] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.436022 11737 leader_election.cc:288] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893), 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.436327 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cfd935f8434229942f6c76bc5e0f87" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e" is_pre_election: true
I20260807 20:08:37.436335 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cfd935f8434229942f6c76bc5e0f87" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3" is_pre_election: true
I20260807 20:08:37.436457 11642 raft_consensus.cc:2454] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.436453 11489 raft_consensus.cc:2454] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.436666 11274 leader_election.cc:302] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, 8648c8cc46f2418eb76b429fb2419598; no voters:
I20260807 20:08:37.436820 11737 raft_consensus.cc:2781] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:37.436883 11737 raft_consensus.cc:482] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:37.436906 11737 raft_consensus.cc:3037] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.437770 11737 raft_consensus.cc:504] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.437939 11737 leader_election.cc:288] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Requested vote from peers e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893), 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.438122 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cfd935f8434229942f6c76bc5e0f87" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e"
I20260807 20:08:37.438134 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59cfd935f8434229942f6c76bc5e0f87" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3"
I20260807 20:08:37.438222 11642 raft_consensus.cc:3037] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.438223 11489 raft_consensus.cc:3037] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.438846 11642 raft_consensus.cc:2454] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.438895 11489 raft_consensus.cc:2454] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.439046 11274 leader_election.cc:302] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, 8648c8cc46f2418eb76b429fb2419598; no voters:
I20260807 20:08:37.439148 11737 raft_consensus.cc:2781] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:37.439215 11737 raft_consensus.cc:686] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Becoming Leader. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Running, Role: LEADER
I20260807 20:08:37.439281 11737 consensus_queue.cc:237] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.439937 11177 catalog_manager.cc:5495] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } health_report { overall_health: UNKNOWN } } }
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133316, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.190@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:37.469367 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.461542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.3.15.190:39345)
0807 20:08:37.461628 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.461631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.461742 (+ 111us) server_negotiation.cc:407] Connection header received
0807 20:08:37.461828 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.461829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.461838 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.461960 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.462459 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.462968 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.463692 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.463921 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.463982 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.466291 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:37.466292 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:37.466292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:37.466306 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:37.467814 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.468375 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.468377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.468377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.468470 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.468711 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.468713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.468713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.468869 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:37.468987 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.469148 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.469237 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 20:08:37.469434 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.461476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39345 (local address 127.0.0.1:43554)
0807 20:08:37.461655 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.461667 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.461764 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.461970 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.461971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.462309 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:37.462357 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.462359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.463049 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.463050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.463599 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.463601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.463979 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.463980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.464034 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.464659 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:37.464664 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:37.466201 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:37.467895 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:37.467903 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:37.467906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:37.468259 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:37.468570 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:37.468571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:37.468571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:37.468645 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:37.468976 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:37.468984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:37.469034 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.469123 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.469294 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":68,"thread_start_us":96,"threads_started":1}
I20260807 20:08:37.471462 11177 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:43554:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 20:08:37.471679 11177 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:37.473227 11469 tablet_service.cc:1467] Processing CreateTablet for tablet 2ec715b99ca544a396a0759ff776b1c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e225420b83b74a44b9b67197a03b8b11]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:37.473215 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 2ec715b99ca544a396a0759ff776b1c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e225420b83b74a44b9b67197a03b8b11]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:37.473390 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ec715b99ca544a396a0759ff776b1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.473388 11469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ec715b99ca544a396a0759ff776b1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.475008 11723 tablet_bootstrap.cc:492] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: Bootstrap starting.
I20260807 20:08:37.475004 11719 tablet_bootstrap.cc:492] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: Bootstrap starting.
I20260807 20:08:37.475081 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 2ec715b99ca544a396a0759ff776b1c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e225420b83b74a44b9b67197a03b8b11]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 20:08:37.475227 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ec715b99ca544a396a0759ff776b1c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.475394 11723 tablet_bootstrap.cc:654] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.475390 11719 tablet_bootstrap.cc:654] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.475976 11719 tablet_bootstrap.cc:492] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: No bootstrap required, opened a new log
I20260807 20:08:37.476042 11719 ts_tablet_manager.cc:1397] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:37.476119 11723 tablet_bootstrap.cc:492] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: No bootstrap required, opened a new log
I20260807 20:08:37.476174 11723 ts_tablet_manager.cc:1397] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:37.476181 11719 raft_consensus.cc:348] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.476285 11719 raft_consensus.cc:374] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.476316 11719 raft_consensus.cc:729] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.476369 11719 consensus_queue.cc:260] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.476380 11723 raft_consensus.cc:348] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.476449 11723 raft_consensus.cc:374] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.476477 11723 raft_consensus.cc:729] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3403af13a55471ba6a4fbf9954175f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.476492 11719 ts_tablet_manager.cc:1428] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 20:08:37.476517 11723 consensus_queue.cc:260] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.476589 11737 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.476647 11723 ts_tablet_manager.cc:1428] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:37.476686 11737 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.476830 11726 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.476886 11718 tablet_bootstrap.cc:492] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: Bootstrap starting.
I20260807 20:08:37.476934 11726 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.477372 11718 tablet_bootstrap.cc:654] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.478127 11718 tablet_bootstrap.cc:492] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: No bootstrap required, opened a new log
I20260807 20:08:37.478190 11718 ts_tablet_manager.cc:1397] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:37.478327 11718 raft_consensus.cc:348] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.478400 11718 raft_consensus.cc:374] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.478423 11718 raft_consensus.cc:729] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840caf72a91a4bacb49f361f42494d2e, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.478479 11718 consensus_queue.cc:260] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.478567 11718 ts_tablet_manager.cc:1428] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.478653 11725 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.478790 11725 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.627480 11737 raft_consensus.cc:482] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:37.627614 11737 raft_consensus.cc:504] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.627815 11737 leader_election.cc:288] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893), 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.628049 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec715b99ca544a396a0759ff776b1c7" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e" is_pre_election: true
I20260807 20:08:37.628046 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec715b99ca544a396a0759ff776b1c7" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3" is_pre_election: true
I20260807 20:08:37.628144 11642 raft_consensus.cc:2454] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.628170 11489 raft_consensus.cc:2454] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.628312 11274 leader_election.cc:302] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, 8648c8cc46f2418eb76b429fb2419598; no voters:
I20260807 20:08:37.628413 11737 raft_consensus.cc:2781] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:37.628469 11737 raft_consensus.cc:482] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:37.628500 11737 raft_consensus.cc:3037] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.629318 11737 raft_consensus.cc:504] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.629494 11737 leader_election.cc:288] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Requested vote from peers e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893), 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.629673 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec715b99ca544a396a0759ff776b1c7" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e"
I20260807 20:08:37.629678 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ec715b99ca544a396a0759ff776b1c7" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3"
I20260807 20:08:37.629752 11642 raft_consensus.cc:3037] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.629769 11489 raft_consensus.cc:3037] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.630409 11642 raft_consensus.cc:2454] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.630545 11489 raft_consensus.cc:2454] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.630753 11273 leader_election.cc:302] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8648c8cc46f2418eb76b429fb2419598, e3403af13a55471ba6a4fbf9954175f3; no voters:
I20260807 20:08:37.630861 11737 raft_consensus.cc:2781] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:37.630949 11737 raft_consensus.cc:686] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Becoming Leader. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Running, Role: LEADER
I20260807 20:08:37.631016 11737 consensus_queue.cc:237] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.631307 11737 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: TxnStatusTablet state changed. Reason: New leader 8648c8cc46f2418eb76b429fb2419598. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.631388 11737 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:37.631778 11760 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 20:08:37.631877 11760 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 20:08:37.631785 11177 catalog_manager.cc:5495] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } health_report { overall_health: UNKNOWN } } }
I20260807 20:08:37.659180 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.656825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.3.15.129:40871)
0807 20:08:37.656896 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.656898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.656908 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:37.656989 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.656990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.657000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.657051 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.657201 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.657682 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.658449 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.658652 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.658730 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.658906 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.658949 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.659005 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.659080 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.659104 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:08:37.659198 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.656730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40871 (local address 127.0.0.1:60126)
0807 20:08:37.656836 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.656854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.656911 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.657059 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.657060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.657061 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:37.657129 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.657132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.657770 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.657771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.658357 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.658359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.658729 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.658730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.658803 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.658818 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.659023 (+ 205us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.659034 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.659077 (+ 43us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.659130 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 20:08:37.660235 11642 raft_consensus.cc:1261] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.660295 11489 raft_consensus.cc:1261] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.660495 11728 consensus_queue.cc:1035] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.660600 11737 consensus_queue.cc:1035] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.661159 11726 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: TxnStatusTablet state changed. Reason: New leader 8648c8cc46f2418eb76b429fb2419598. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.661257 11726 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.661347 11725 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: TxnStatusTablet state changed. Reason: New leader 8648c8cc46f2418eb76b429fb2419598. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.661438 11725 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.662031 11728 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.662114 11728 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:37.662163 11728 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.662205 11728 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 8648c8cc46f2418eb76b429fb2419598: This TxnStatusTablet replica's current role is: LEADER
I20260807 20:08:37.662266 11725 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.662356 11725 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P 840caf72a91a4bacb49f361f42494d2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.662523 11726 tablet_replica.cc:439] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } } }
I20260807 20:08:37.662612 11726 tablet_replica.cc:442] T 2ec715b99ca544a396a0759ff776b1c7 P e3403af13a55471ba6a4fbf9954175f3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 20:08:37.662582 11733 mvcc.cc:205] Tried to move back new op lower bound from 7316002069134635008 to 7316002069017141248. Current Snapshot: MvccSnapshot[applied={T|T < 7316002069134635008}]
I20260807 20:08:37.668973 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.666316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:43013 (local address 127.3.15.129:40871)
0807 20:08:37.666355 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.666357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.666491 (+ 134us) server_negotiation.cc:407] Connection header received
0807 20:08:37.666554 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.666555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.666564 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.666624 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.666741 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.667321 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.668178 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.668435 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.668505 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.668683 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.668715 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.668757 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.668831 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.668874 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":17}
I20260807 20:08:37.668974 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.666250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40871 (local address 127.3.15.129:43013)
0807 20:08:37.666426 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.666435 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.666505 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.666629 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.666630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.666632 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:37.666690 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.666691 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.667395 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.667396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.668084 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.668087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.668519 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.668520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.668568 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.668591 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.668768 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.668769 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.668829 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.668895 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":20,"thread_start_us":76,"threads_started":1}
I20260807 20:08:37.671399 11642 raft_consensus.cc:1261] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.671399 11489 raft_consensus.cc:1261] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.671622 11728 consensus_queue.cc:1035] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.671742 11722 consensus_queue.cc:1035] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.686527 11178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.686645 11178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.687424 11178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Sending DeleteTablet for 3 replicas of tablet 59cfd935f8434229942f6c76bc5e0f87
I20260807 20:08:37.687769 11618 tablet_service.cc:1514] Processing DeleteTablet for tablet 59cfd935f8434229942f6c76bc5e0f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:55836
I20260807 20:08:37.687779 11469 tablet_service.cc:1514] Processing DeleteTablet for tablet 59cfd935f8434229942f6c76bc5e0f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41360
I20260807 20:08:37.687829 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 59cfd935f8434229942f6c76bc5e0f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:60110
I20260807 20:08:37.687881 11745 tablet_replica.cc:330] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: stopping tablet replica
I20260807 20:08:37.687898 11743 tablet_replica.cc:330] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: stopping tablet replica
I20260807 20:08:37.687920 11744 tablet_replica.cc:330] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: stopping tablet replica
I20260807 20:08:37.687927 11745 raft_consensus.cc:2229] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.687953 11743 raft_consensus.cc:2229] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.687964 11744 raft_consensus.cc:2229] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:37.687978 11745 raft_consensus.cc:2258] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.688000 11743 raft_consensus.cc:2258] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.688032 11744 raft_consensus.cc:2258] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.688328 11745 ts_tablet_manager.cc:1905] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.688337 11743 ts_tablet_manager.cc:1905] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.688408 11744 ts_tablet_manager.cc:1905] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.690095 11744 ts_tablet_manager.cc:1918] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:37.690109 11743 ts_tablet_manager.cc:1918] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:37.690164 11744 log.cc:1196] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/59cfd935f8434229942f6c76bc5e0f87
I20260807 20:08:37.690167 11743 log.cc:1196] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/59cfd935f8434229942f6c76bc5e0f87
I20260807 20:08:37.690223 11745 ts_tablet_manager.cc:1918] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:37.690281 11745 log.cc:1196] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/59cfd935f8434229942f6c76bc5e0f87
I20260807 20:08:37.690412 11743 ts_tablet_manager.cc:1939] T 59cfd935f8434229942f6c76bc5e0f87 P 840caf72a91a4bacb49f361f42494d2e: Deleting consensus metadata
I20260807 20:08:37.690459 11744 ts_tablet_manager.cc:1939] T 59cfd935f8434229942f6c76bc5e0f87 P 8648c8cc46f2418eb76b429fb2419598: Deleting consensus metadata
I20260807 20:08:37.690531 11745 ts_tablet_manager.cc:1939] T 59cfd935f8434229942f6c76bc5e0f87 P e3403af13a55471ba6a4fbf9954175f3: Deleting consensus metadata
I20260807 20:08:37.690752 11165 catalog_manager.cc:4841] TS 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651): tablet 59cfd935f8434229942f6c76bc5e0f87 (table test-table [id=51d55f01ec20466c89422b09d2a39315]) successfully deleted
I20260807 20:08:37.690946 11164 catalog_manager.cc:4841] TS e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893): tablet 59cfd935f8434229942f6c76bc5e0f87 (table test-table [id=51d55f01ec20466c89422b09d2a39315]) successfully deleted
I20260807 20:08:37.691012 11165 catalog_manager.cc:4841] TS 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871): tablet 59cfd935f8434229942f6c76bc5e0f87 (table test-table [id=51d55f01ec20466c89422b09d2a39315]) successfully deleted
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 20:08:37.714815 11230 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:37.708633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.3.15.190:39345)
0807 20:08:37.708707 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.708709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.708819 (+ 110us) server_negotiation.cc:407] Connection header received
0807 20:08:37.708927 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.708929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.708951 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.709024 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.709627 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.710439 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.711253 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.711540 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.711622 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.714749 (+ 3127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43564: BlockingRecv error: recv got EOF from 127.0.0.1:43564 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 20:08:37 dist-test-slave-1x4g krb5kdc[11132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM
I20260807 20:08:37.723728 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.715634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43570 (local address 127.3.15.190:39345)
0807 20:08:37.715676 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.715678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.715920 (+ 242us) server_negotiation.cc:407] Connection header received
0807 20:08:37.715990 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.715991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.716000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.716048 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:37.716649 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.717171 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.718192 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.718362 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.718412 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.720761 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:37.720763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:37.720763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:37.720771 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:37.722495 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.722907 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.722908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.722908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.722971 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:37.723191 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:37.723192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:37.723193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:37.723304 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:37.723411 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.723534 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.723621 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:37.725459 11178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:37.725654 11178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:37.727149 11469 tablet_service.cc:1467] Processing CreateTablet for tablet 45cfab56d5524391a394310e1bb21264 (DEFAULT_TABLE table=test-table [id=4d708b8b382c489ca4c2c4945149f85b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.727226 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 45cfab56d5524391a394310e1bb21264 (DEFAULT_TABLE table=test-table [id=4d708b8b382c489ca4c2c4945149f85b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.727298 11469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45cfab56d5524391a394310e1bb21264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.727288 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 45cfab56d5524391a394310e1bb21264 (DEFAULT_TABLE table=test-table [id=4d708b8b382c489ca4c2c4945149f85b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.727370 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45cfab56d5524391a394310e1bb21264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.727399 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45cfab56d5524391a394310e1bb21264. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.728724 11719 tablet_bootstrap.cc:492] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Bootstrap starting.
I20260807 20:08:37.728758 11723 tablet_bootstrap.cc:492] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Bootstrap starting.
I20260807 20:08:37.729090 11719 tablet_bootstrap.cc:654] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.729270 11723 tablet_bootstrap.cc:654] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.729614 11718 tablet_bootstrap.cc:492] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Bootstrap starting.
I20260807 20:08:37.729745 11719 tablet_bootstrap.cc:492] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: No bootstrap required, opened a new log
I20260807 20:08:37.729789 11719 ts_tablet_manager.cc:1397] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:37.729923 11718 tablet_bootstrap.cc:654] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.729890 11719 raft_consensus.cc:348] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.729964 11719 raft_consensus.cc:374] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.729986 11719 raft_consensus.cc:729] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.730031 11719 consensus_queue.cc:260] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.730127 11719 ts_tablet_manager.cc:1428] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.730753 11723 tablet_bootstrap.cc:492] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: No bootstrap required, opened a new log
I20260807 20:08:37.730810 11723 ts_tablet_manager.cc:1397] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:08:37.730973 11723 raft_consensus.cc:348] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.731060 11723 raft_consensus.cc:374] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.731101 11723 raft_consensus.cc:729] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3403af13a55471ba6a4fbf9954175f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.731149 11723 consensus_queue.cc:260] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.731237 11723 ts_tablet_manager.cc:1428] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.731637 11718 tablet_bootstrap.cc:492] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: No bootstrap required, opened a new log
I20260807 20:08:37.731691 11718 ts_tablet_manager.cc:1397] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:08:37.731844 11718 raft_consensus.cc:348] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.732065 11718 raft_consensus.cc:374] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.732143 11718 raft_consensus.cc:729] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840caf72a91a4bacb49f361f42494d2e, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.732221 11718 consensus_queue.cc:260] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.732365 11718 ts_tablet_manager.cc:1428] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:37.838754 11728 raft_consensus.cc:482] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:37.838871 11728 raft_consensus.cc:504] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.839042 11728 leader_election.cc:288] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893), 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.839329 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cfab56d5524391a394310e1bb21264" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e" is_pre_election: true
I20260807 20:08:37.839329 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cfab56d5524391a394310e1bb21264" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3" is_pre_election: true
I20260807 20:08:37.839438 11642 raft_consensus.cc:2454] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.839438 11489 raft_consensus.cc:2454] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 0.
I20260807 20:08:37.839618 11273 leader_election.cc:302] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8648c8cc46f2418eb76b429fb2419598, e3403af13a55471ba6a4fbf9954175f3; no voters:
I20260807 20:08:37.839737 11728 raft_consensus.cc:2781] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:37.839781 11728 raft_consensus.cc:482] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:37.839800 11728 raft_consensus.cc:3037] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.840579 11728 raft_consensus.cc:504] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.840730 11728 leader_election.cc:288] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Requested vote from peers e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893), 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651)
I20260807 20:08:37.840924 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cfab56d5524391a394310e1bb21264" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3"
I20260807 20:08:37.840937 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45cfab56d5524391a394310e1bb21264" candidate_uuid: "8648c8cc46f2418eb76b429fb2419598" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "840caf72a91a4bacb49f361f42494d2e"
I20260807 20:08:37.841012 11642 raft_consensus.cc:3037] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.841012 11489 raft_consensus.cc:3037] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.841768 11642 raft_consensus.cc:2454] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.841768 11489 raft_consensus.cc:2454] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8648c8cc46f2418eb76b429fb2419598 in term 1.
I20260807 20:08:37.842000 11274 leader_election.cc:302] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, 8648c8cc46f2418eb76b429fb2419598; no voters:
I20260807 20:08:37.842150 11728 raft_consensus.cc:2781] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:37.842218 11728 raft_consensus.cc:686] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Becoming Leader. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Running, Role: LEADER
I20260807 20:08:37.842305 11728 consensus_queue.cc:237] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.843007 11178 catalog_manager.cc:5495] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "8648c8cc46f2418eb76b429fb2419598" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } health_report { overall_health: UNKNOWN } } }
I20260807 20:08:37.876222 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.873893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.3.15.129:40871)
0807 20:08:37.873974 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.873976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.873984 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:37.874043 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.874043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.874053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.874111 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.874237 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.874709 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.875514 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.875696 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.875747 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.875873 (+ 126us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.875899 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.875936 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.876006 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.876025 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 20:08:37.877242 11489 raft_consensus.cc:1261] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.877241 11642 raft_consensus.cc:1261] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Refusing update from remote peer 8648c8cc46f2418eb76b429fb2419598: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.877522 11722 consensus_queue.cc:1035] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.877692 11728 consensus_queue.cc:1035] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.880671 11178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.880764 11178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.881445 11178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Sending DeleteTablet for 3 replicas of tablet 45cfab56d5524391a394310e1bb21264
I20260807 20:08:37.881721 11469 tablet_service.cc:1514] Processing DeleteTablet for tablet 45cfab56d5524391a394310e1bb21264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41360
I20260807 20:08:37.881757 11618 tablet_service.cc:1514] Processing DeleteTablet for tablet 45cfab56d5524391a394310e1bb21264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:55836
I20260807 20:08:37.881769 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 45cfab56d5524391a394310e1bb21264 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:60110
I20260807 20:08:37.881812 11745 tablet_replica.cc:330] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: stopping tablet replica
I20260807 20:08:37.881839 11743 tablet_replica.cc:330] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: stopping tablet replica
I20260807 20:08:37.881862 11745 raft_consensus.cc:2229] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.881858 11744 tablet_replica.cc:330] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: stopping tablet replica
I20260807 20:08:37.881878 11743 raft_consensus.cc:2229] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.881907 11745 raft_consensus.cc:2258] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.881918 11744 raft_consensus.cc:2229] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:37.881929 11743 raft_consensus.cc:2258] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.881971 11744 raft_consensus.cc:2258] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.881948 11178 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:08:37.882091 11178 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:08:37.882282 11745 ts_tablet_manager.cc:1905] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.882526 11744 ts_tablet_manager.cc:1905] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.882652 11743 ts_tablet_manager.cc:1905] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.884023 11745 ts_tablet_manager.cc:1918] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:37.884083 11745 log.cc:1196] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/45cfab56d5524391a394310e1bb21264
I20260807 20:08:37.884065 11744 ts_tablet_manager.cc:1918] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:37.884137 11744 log.cc:1196] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/45cfab56d5524391a394310e1bb21264
I20260807 20:08:37.884220 11743 ts_tablet_manager.cc:1918] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:08:37.884274 11743 log.cc:1196] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/45cfab56d5524391a394310e1bb21264
I20260807 20:08:37.884334 11745 ts_tablet_manager.cc:1939] T 45cfab56d5524391a394310e1bb21264 P e3403af13a55471ba6a4fbf9954175f3: Deleting consensus metadata
I20260807 20:08:37.884375 11744 ts_tablet_manager.cc:1939] T 45cfab56d5524391a394310e1bb21264 P 8648c8cc46f2418eb76b429fb2419598: Deleting consensus metadata
I20260807 20:08:37.884518 11743 ts_tablet_manager.cc:1939] T 45cfab56d5524391a394310e1bb21264 P 840caf72a91a4bacb49f361f42494d2e: Deleting consensus metadata
I20260807 20:08:37.884953 11321 tablet_service.cc:1467] Processing CreateTablet for tablet 112f144f821a42fe869bacb2a1d16c68 (DEFAULT_TABLE table=test-table [id=581de807460f4103835dd5c18ab40dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.884963 11618 tablet_service.cc:1467] Processing CreateTablet for tablet 112f144f821a42fe869bacb2a1d16c68 (DEFAULT_TABLE table=test-table [id=581de807460f4103835dd5c18ab40dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.885077 11321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112f144f821a42fe869bacb2a1d16c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.885110 11618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112f144f821a42fe869bacb2a1d16c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.885933 11718 tablet_bootstrap.cc:492] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Bootstrap starting.
I20260807 20:08:37.885998 11719 tablet_bootstrap.cc:492] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Bootstrap starting.
I20260807 20:08:37.886075 11164 catalog_manager.cc:4841] TS e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893): tablet 45cfab56d5524391a394310e1bb21264 (table test-table [id=4d708b8b382c489ca4c2c4945149f85b]) successfully deleted
I20260807 20:08:37.886262 11719 tablet_bootstrap.cc:654] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.886299 11469 tablet_service.cc:1467] Processing CreateTablet for tablet 112f144f821a42fe869bacb2a1d16c68 (DEFAULT_TABLE table=test-table [id=581de807460f4103835dd5c18ab40dfa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:08:37.886435 11469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112f144f821a42fe869bacb2a1d16c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:37.886519 11718 tablet_bootstrap.cc:654] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.886744 11719 tablet_bootstrap.cc:492] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: No bootstrap required, opened a new log
I20260807 20:08:37.886776 11719 ts_tablet_manager.cc:1397] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260807 20:08:37.886857 11719 raft_consensus.cc:348] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.886901 11719 raft_consensus.cc:374] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.886914 11719 raft_consensus.cc:729] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8648c8cc46f2418eb76b429fb2419598, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.886971 11719 consensus_queue.cc:260] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.887097 11719 ts_tablet_manager.cc:1428] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.887363 11718 tablet_bootstrap.cc:492] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: No bootstrap required, opened a new log
I20260807 20:08:37.887418 11718 ts_tablet_manager.cc:1397] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:37.887550 11718 raft_consensus.cc:348] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.887643 11718 raft_consensus.cc:374] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.887669 11718 raft_consensus.cc:729] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840caf72a91a4bacb49f361f42494d2e, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.887712 11718 consensus_queue.cc:260] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.887758 11723 tablet_bootstrap.cc:492] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Bootstrap starting.
I20260807 20:08:37.887816 11718 ts_tablet_manager.cc:1428] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.888191 11723 tablet_bootstrap.cc:654] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:37.888933 11165 catalog_manager.cc:4841] TS 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651): tablet 45cfab56d5524391a394310e1bb21264 (table test-table [id=4d708b8b382c489ca4c2c4945149f85b]) successfully deleted
I20260807 20:08:37.889218 11723 tablet_bootstrap.cc:492] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: No bootstrap required, opened a new log
I20260807 20:08:37.889281 11723 ts_tablet_manager.cc:1397] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:08:37.889276 11165 catalog_manager.cc:4841] TS 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871): tablet 45cfab56d5524391a394310e1bb21264 (table test-table [id=4d708b8b382c489ca4c2c4945149f85b]) successfully deleted
I20260807 20:08:37.889436 11723 raft_consensus.cc:348] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.889510 11723 raft_consensus.cc:374] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:37.889544 11723 raft_consensus.cc:729] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3403af13a55471ba6a4fbf9954175f3, State: Initialized, Role: FOLLOWER
I20260807 20:08:37.889597 11723 consensus_queue.cc:260] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.889686 11723 ts_tablet_manager.cc:1428] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:37.910352 11725 raft_consensus.cc:482] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:08:37.910450 11725 raft_consensus.cc:504] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.910923 11725 leader_election.cc:288] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871), e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893)
I20260807 20:08:37.914127 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.911269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:49173 (local address 127.3.15.129:40871)
0807 20:08:37.911324 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.911326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.911335 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:37.911414 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.911415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.911427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.911485 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.911814 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.912318 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.913367 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.913739 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.913827 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.913837 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.914004 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.914042 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:08:37.914155 11796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.911080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40871 (local address 127.3.15.131:49173)
0807 20:08:37.911268 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.911281 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.911339 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.911492 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.911492 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.911494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.911726 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.911729 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.912480 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.912481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.913280 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.913282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.913852 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.913853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.913918 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.913922 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.913996 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.914038 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":65,"threads_started":1}
I20260807 20:08:37.914156 11797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.911190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:49495 (local address 127.3.15.130:41893)
0807 20:08:37.911336 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.911338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.911410 (+ 72us) server_negotiation.cc:407] Connection header received
0807 20:08:37.911479 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.911480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.911491 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.911573 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:08:37.911726 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.912244 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.913333 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.913735 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.913836 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.913845 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.914028 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.914062 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
I20260807 20:08:37.914166 11795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.911106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:41893 (local address 127.3.15.131:49495)
0807 20:08:37.911344 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.911354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.911420 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.911574 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.911575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.911577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:08:37.911658 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.911660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.912329 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.912330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.913241 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.913244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.913848 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.913850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.913940 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.913943 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.914008 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.914053 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":14,"thread_start_us":85,"threads_started":1}
I20260807 20:08:37.914458 11342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112f144f821a42fe869bacb2a1d16c68" candidate_uuid: "840caf72a91a4bacb49f361f42494d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8648c8cc46f2418eb76b429fb2419598" is_pre_election: true
I20260807 20:08:37.914614 11342 raft_consensus.cc:2454] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 840caf72a91a4bacb49f361f42494d2e in term 0.
I20260807 20:08:37.914707 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112f144f821a42fe869bacb2a1d16c68" candidate_uuid: "840caf72a91a4bacb49f361f42494d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3" is_pre_election: true
I20260807 20:08:37.914796 11489 raft_consensus.cc:2454] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 840caf72a91a4bacb49f361f42494d2e in term 0.
I20260807 20:08:37.914832 11569 leader_election.cc:302] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, 8648c8cc46f2418eb76b429fb2419598; no voters:
I20260807 20:08:37.914997 11725 raft_consensus.cc:2781] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:08:37.915047 11725 raft_consensus.cc:482] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:08:37.915069 11725 raft_consensus.cc:3037] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.915783 11725 raft_consensus.cc:504] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.915920 11725 leader_election.cc:288] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [CANDIDATE]: Term 1 election: Requested vote from peers 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871), e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893)
I20260807 20:08:37.916069 11489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112f144f821a42fe869bacb2a1d16c68" candidate_uuid: "840caf72a91a4bacb49f361f42494d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3403af13a55471ba6a4fbf9954175f3"
I20260807 20:08:37.916134 11489 raft_consensus.cc:3037] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.916183 11342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112f144f821a42fe869bacb2a1d16c68" candidate_uuid: "840caf72a91a4bacb49f361f42494d2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8648c8cc46f2418eb76b429fb2419598"
I20260807 20:08:37.916268 11342 raft_consensus.cc:3037] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:37.916828 11489 raft_consensus.cc:2454] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 840caf72a91a4bacb49f361f42494d2e in term 1.
I20260807 20:08:37.917001 11568 leader_election.cc:302] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 840caf72a91a4bacb49f361f42494d2e, e3403af13a55471ba6a4fbf9954175f3; no voters:
I20260807 20:08:37.917085 11725 raft_consensus.cc:2781] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:37.917119 11342 raft_consensus.cc:2454] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 840caf72a91a4bacb49f361f42494d2e in term 1.
I20260807 20:08:37.917188 11725 raft_consensus.cc:686] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 1 LEADER]: Becoming Leader. State: Replica: 840caf72a91a4bacb49f361f42494d2e, State: Running, Role: LEADER
I20260807 20:08:37.917274 11725 consensus_queue.cc:237] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } }
I20260807 20:08:37.917979 11178 catalog_manager.cc:5495] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 840caf72a91a4bacb49f361f42494d2e (127.3.15.131). New cstate: current_term: 1 leader_uuid: "840caf72a91a4bacb49f361f42494d2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "840caf72a91a4bacb49f361f42494d2e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46651 } health_report { overall_health: HEALTHY } } }
I20260807 20:08:37.932024 11800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.929429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.3.15.131:46651)
0807 20:08:37.929628 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.929630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.929637 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:37.929673 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.929674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.929695 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.929800 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.929910 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.930434 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.931161 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.931380 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.931454 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.931642 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.931761 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.931845 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.931896 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.931917 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":114,"threads_started":1}
I20260807 20:08:37.936861 11717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.934426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:41079 (local address 127.3.15.129:40871)
0807 20:08:37.934506 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.934509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.934609 (+ 100us) server_negotiation.cc:407] Connection header received
0807 20:08:37.934674 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.934675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.934685 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.934728 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.934883 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.935340 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.936129 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.936334 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.936409 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.936589 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.936639 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.936701 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.936757 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.936790 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 20:08:37.936900 11803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.934330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40871 (local address 127.3.15.131:41079)
0807 20:08:37.934529 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.934546 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.934618 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.934740 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.934741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.934742 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:37.934803 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.934804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.935421 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.935422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.936035 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.936038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.936409 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.936410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.936487 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.936500 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.936703 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.936705 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.936753 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.936817 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":92,"threads_started":1}
I20260807 20:08:37.939347 11489 raft_consensus.cc:1261] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Refusing update from remote peer 840caf72a91a4bacb49f361f42494d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.939368 11342 raft_consensus.cc:1261] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Refusing update from remote peer 840caf72a91a4bacb49f361f42494d2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:08:37.939627 11725 consensus_queue.cc:1035] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3403af13a55471ba6a4fbf9954175f3" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41893 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.939805 11798 consensus_queue.cc:1035] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8648c8cc46f2418eb76b429fb2419598" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:08:37.947794 11800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.945579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:42199 (local address 127.3.15.131:46651)
0807 20:08:37.945645 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:37.945648 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:37.945656 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:37.945680 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:37.945680 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:37.945690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:37.945742 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:08:37.945828 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.946269 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.947068 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.947379 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.947454 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.947599 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.947635 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.947684 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:37.947728 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 20:08:37.947746 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:08:37.947817 11768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:37.945355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46651 (local address 127.3.15.129:42199)
0807 20:08:37.945427 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:37.945439 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:37.945502 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:37.945748 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:37.945749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:37.945750 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 20:08:37.945787 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.945788 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.946371 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.946372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.946971 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:37.946973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:37.947453 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:37.947454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:37.947519 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:37.947528 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 20:08:37.947685 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 20:08:37.947686 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 20:08:37.947727 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 20:08:37.947765 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 20:08:37.954396 11178 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.954494 11178 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:08:37.955160 11178 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a76e8900b5c14a45af474eb0001a7ab6: Sending DeleteTablet for 3 replicas of tablet 112f144f821a42fe869bacb2a1d16c68
I20260807 20:08:37.955436 11321 tablet_service.cc:1514] Processing DeleteTablet for tablet 112f144f821a42fe869bacb2a1d16c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:60110
I20260807 20:08:37.955497 11469 tablet_service.cc:1514] Processing DeleteTablet for tablet 112f144f821a42fe869bacb2a1d16c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41360
I20260807 20:08:37.955534 11744 tablet_replica.cc:330] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: stopping tablet replica
I20260807 20:08:37.955581 11745 tablet_replica.cc:330] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: stopping tablet replica
I20260807 20:08:37.955581 11744 raft_consensus.cc:2229] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.955629 11745 raft_consensus.cc:2229] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:08:37.955639 11744 raft_consensus.cc:2258] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.955677 11745 raft_consensus.cc:2258] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.955796 11618 tablet_service.cc:1514] Processing DeleteTablet for tablet 112f144f821a42fe869bacb2a1d16c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:08:37 UTC) from {username='oryx', principal='oryx/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:55836
I20260807 20:08:37.955900 11743 tablet_replica.cc:330] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: stopping tablet replica
I20260807 20:08:37.955956 11743 raft_consensus.cc:2229] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:08:37.956053 11743 raft_consensus.cc:2258] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:08:37.956141 11745 ts_tablet_manager.cc:1905] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.956336 11743 ts_tablet_manager.cc:1905] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.958058 11744 ts_tablet_manager.cc:1905] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:08:37.958216 11745 ts_tablet_manager.cc:1918] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:37.958277 11745 log.cc:1196] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/112f144f821a42fe869bacb2a1d16c68
I20260807 20:08:37.958316 11743 ts_tablet_manager.cc:1918] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:37.958384 11743 log.cc:1196] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/112f144f821a42fe869bacb2a1d16c68
I20260807 20:08:37.958526 11745 ts_tablet_manager.cc:1939] T 112f144f821a42fe869bacb2a1d16c68 P e3403af13a55471ba6a4fbf9954175f3: Deleting consensus metadata
I20260807 20:08:37.958631 11743 ts_tablet_manager.cc:1939] T 112f144f821a42fe869bacb2a1d16c68 P 840caf72a91a4bacb49f361f42494d2e: Deleting consensus metadata
I20260807 20:08:37.958861 11164 catalog_manager.cc:4841] TS e3403af13a55471ba6a4fbf9954175f3 (127.3.15.130:41893): tablet 112f144f821a42fe869bacb2a1d16c68 (table test-table [id=581de807460f4103835dd5c18ab40dfa]) successfully deleted
I20260807 20:08:37.959512 11744 ts_tablet_manager.cc:1918] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 20:08:37.959563 11744 log.cc:1196] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/112f144f821a42fe869bacb2a1d16c68
I20260807 20:08:37.959784 11744 ts_tablet_manager.cc:1939] T 112f144f821a42fe869bacb2a1d16c68 P 8648c8cc46f2418eb76b429fb2419598: Deleting consensus metadata
I20260807 20:08:37.960129 11165 catalog_manager.cc:4841] TS 8648c8cc46f2418eb76b429fb2419598 (127.3.15.129:40871): tablet 112f144f821a42fe869bacb2a1d16c68 (table test-table [id=581de807460f4103835dd5c18ab40dfa]) successfully deleted
I20260807 20:08:37.960216 11165 catalog_manager.cc:4841] TS 840caf72a91a4bacb49f361f42494d2e (127.3.15.131:46651): tablet 112f144f821a42fe869bacb2a1d16c68 (table test-table [id=581de807460f4103835dd5c18ab40dfa]) successfully deleted
I20260807 20:08:37.960343 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 11254
I20260807 20:08:37.970746 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 11399
I20260807 20:08:37.978868 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 11550
I20260807 20:08:37.985837 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 11145
2026-08-07T20:08:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2714 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:38 dist-test-slave-1x4g krb5kdc[11810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:38 dist-test-slave-1x4g krb5kdc[11810](info): set up 2 sockets
Aug 07 20:08:38 dist-test-slave-1x4g krb5kdc[11810](info): commencing operation
krb5kdc: starting...
W20260807 20:08:39.082086 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:39 dist-test-slave-1x4g krb5kdc[11810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:39.211616 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42811
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:40473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:42811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:39.294770 11827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:39.295013 11827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:39.295049 11827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:39.296175 11827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:39.296243 11827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:39.296270 11827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:39.296303 11827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:39.296326 11827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:39.297922 11827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:42811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:39.298220 11827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:39.298448 11827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:39.303237 11832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:39.303326 11835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:39.303260 11833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:39.303524 11827 server_base.cc:1034] running on GCE node
I20260807 20:08:39.303781 11827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:39.304051 11827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:39.305207 11827 hybrid_clock.cc:648] HybridClock initialized: now 1786133319305177 us; error 50 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T20:08:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1321 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:39 dist-test-slave-1x4g krb5kdc[11840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:39 dist-test-slave-1x4g krb5kdc[11840](info): set up 2 sockets
Aug 07 20:08:39 dist-test-slave-1x4g krb5kdc[11840](info): commencing operation
krb5kdc: starting...
W20260807 20:08:40.378485 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:40 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:40.507929 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39275
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:40.601711 11853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:40.601938 11853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:40.601961 11853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:40.603264 11853 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:40.603309 11853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:40.603324 11853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:40.603338 11853 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:40.603350 11853 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:40.604949 11853 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:40.605428 11853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:40.605749 11853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:40.611590 11858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:40.611804 11853 server_base.cc:1034] running on GCE node
W20260807 20:08:40.611595 11861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:40.612264 11859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:40.612576 11853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:40.612970 11853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:40.614156 11853 hybrid_clock.cc:648] HybridClock initialized: now 1786133320614142 us; error 30 us; skew 500 ppm
Aug 07 20:08:40 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:40.617641 11853 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:40.618252 11853 webserver.cc:460] Webserver started at http://127.3.15.190:32861/ using document root <none> and password file <none>
I20260807 20:08:40.618558 11853 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:40.618636 11853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:40.618795 11853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:40.620381 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "aa08cd3d928a4785ba82358625f0a8b7"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "af830aeb30d040a491111312c9f74ef3"
server_key_iv: "e8203f4dfbc477da1f57e2b1950a66b0"
server_key_version: "encryptionkey@0"
I20260807 20:08:40.620894 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "aa08cd3d928a4785ba82358625f0a8b7"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "af830aeb30d040a491111312c9f74ef3"
server_key_iv: "e8203f4dfbc477da1f57e2b1950a66b0"
server_key_version: "encryptionkey@0"
I20260807 20:08:40.622817 11853 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 20:08:40.623795 11868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:40.624019 11853 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:40.624122 11853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "aa08cd3d928a4785ba82358625f0a8b7"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "af830aeb30d040a491111312c9f74ef3"
server_key_iv: "e8203f4dfbc477da1f57e2b1950a66b0"
server_key_version: "encryptionkey@0"
I20260807 20:08:40.624213 11853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:40.638545 11853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:40.640256 11853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:40.640362 11853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:40.644475 11853 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39275
I20260807 20:08:40.644506 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39275 every 8 connection(s)
I20260807 20:08:40.644963 11853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:40.645867 11926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:40.648123 11926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Bootstrap starting.
I20260807 20:08:40.648701 11926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:40.649019 11926 log.cc:824] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:40.649734 11926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: No bootstrap required, opened a new log
I20260807 20:08:40.651261 11926 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:40.651423 11926 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:40.651472 11926 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa08cd3d928a4785ba82358625f0a8b7, State: Initialized, Role: FOLLOWER
I20260807 20:08:40.651621 11926 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:40.651703 11926 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:40.651747 11926 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:40.651799 11926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:40.652298 11926 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:40.652415 11926 leader_election.cc:302] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa08cd3d928a4785ba82358625f0a8b7; no voters:
I20260807 20:08:40.652598 11926 leader_election.cc:288] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:40.652693 11929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:40.652884 11929 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 LEADER]: Becoming Leader. State: Replica: aa08cd3d928a4785ba82358625f0a8b7, State: Running, Role: LEADER
I20260807 20:08:40.652909 11926 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:40.652981 11929 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:40.653505 11931 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa08cd3d928a4785ba82358625f0a8b7. Latest consensus state: current_term: 1 leader_uuid: "aa08cd3d928a4785ba82358625f0a8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } } }
I20260807 20:08:40.653609 11931 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:40.653661 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 11853
I20260807 20:08:40.653512 11930 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa08cd3d928a4785ba82358625f0a8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } } }
I20260807 20:08:40.653790 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:40.653851 11930 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:40.654009 3134 external_mini_cluster.cc:1428] Setting key 85a920c11afa6a8ebb3b3938e3dd64d9
I20260807 20:08:40.654498 11933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:40.655086 11933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 20:08:40 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:40.662876 11933 catalog_manager.cc:1347] Generated new cluster ID: e58b611d10754cc69675df56fec67030
I20260807 20:08:40.662956 11933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:40.666055 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:40.654532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38576 (local address 127.3.15.190:39275)
0807 20:08:40.655769 (+ 1237us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:40.655779 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:40.655814 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:08:40.655936 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:40.655942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:40.655998 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:40.656112 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:40.656966 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:40.658389 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:40.659160 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:40.659816 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:40.659930 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:40.662296 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:40.662301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:40.662301 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:40.662313 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:40.664363 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:40.664755 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:40.664759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:40.664759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:40.664903 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:40.665129 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:40.665129 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:40.665130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:40.665453 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:40.665623 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:40.665697 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:08:40.665834 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":76,"threads_started":1}
I20260807 20:08:40.688869 11933 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:40.689632 11933 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:40.694857 11933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Generated new TSK 0
I20260807 20:08:40.695076 11933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:40.733698 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39275
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:40.849426 11954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:40.849675 11954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:40.849718 11954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:40.851013 11954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:40.851081 11954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:40.851131 11954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:40.852895 11954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:40.853175 11954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:40.853479 11954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:40.859125 11962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:40.859190 11959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:40.859134 11960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:40.859328 11954 server_base.cc:1034] running on GCE node
I20260807 20:08:40.859551 11954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:40.859783 11954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:40.860963 11954 hybrid_clock.cc:648] HybridClock initialized: now 1786133320860934 us; error 45 us; skew 500 ppm
Aug 07 20:08:40 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:40.863770 11954 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:40.864253 11954 webserver.cc:460] Webserver started at http://127.3.15.129:32871/ using document root <none> and password file <none>
I20260807 20:08:40.864465 11954 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:40.864511 11954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:40.864600 11954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:40.865722 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "9818e7e5152e4aa5a6acbe9ad7f08b47"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "802d3e685df7e144a403f7a2dd9ffc20"
server_key_iv: "b3cacfd1b00611eb1e8569fe73197a6d"
server_key_version: "encryptionkey@0"
I20260807 20:08:40.866039 11954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "9818e7e5152e4aa5a6acbe9ad7f08b47"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "802d3e685df7e144a403f7a2dd9ffc20"
server_key_iv: "b3cacfd1b00611eb1e8569fe73197a6d"
server_key_version: "encryptionkey@0"
I20260807 20:08:40.867337 11954 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:40.868026 11970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:40.868193 11954 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:40.868279 11954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "9818e7e5152e4aa5a6acbe9ad7f08b47"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "802d3e685df7e144a403f7a2dd9ffc20"
server_key_iv: "b3cacfd1b00611eb1e8569fe73197a6d"
server_key_version: "encryptionkey@0"
I20260807 20:08:40.868335 11954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:40.883181 11954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:40.885115 11954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:40.885226 11954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:40.885396 11954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:40.885740 11954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:40.885778 11954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:40.885812 11954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:40.885867 11954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:40.891973 11954 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40095
I20260807 20:08:40.892050 12090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40095 every 8 connection(s)
I20260807 20:08:40.892365 11954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:40 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133320, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:40.900456 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 11954
I20260807 20:08:40.900571 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:40.900857 3134 external_mini_cluster.cc:1428] Setting key aa07144277ddcb6e8e29dd88f7b5d60a
I20260807 20:08:40.901717 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:40.893099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:41077 (local address 127.3.15.190:39275)
0807 20:08:40.893143 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:40.893145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:40.893322 (+ 177us) server_negotiation.cc:407] Connection header received
0807 20:08:40.893869 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:40.893871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:40.893889 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:40.893950 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:40.894920 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:40.895435 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:40.896447 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:40.896671 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:40.896744 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:40.898781 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:40.898783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:40.898783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:40.898790 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:40.900159 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:40.900579 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:40.900580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:40.900581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:40.900649 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:40.900997 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:40.900998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:40.900999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:40.901156 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:40.901246 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:40.901545 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 20:08:40.901639 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:08:40.901942 12093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:40.893053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:41077)
0807 20:08:40.893203 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:40.893244 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:40.893755 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:40.894019 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:40.894023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:40.894597 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:40.894815 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:40.894820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:40.895539 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:40.895540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:40.896338 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:40.896340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:40.896765 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:40.896767 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:40.896922 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:40.897341 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:40.897349 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:40.898691 (+ 1342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:40.900237 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:40.900243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:40.900245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:40.900473 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:40.900719 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:40.900720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:40.900720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:40.900879 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:40.901256 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:40.901264 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:40.901381 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 20:08:40.901503 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 20:08:40.901687 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260807 20:08:40.902444 12091 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39275
I20260807 20:08:40.902539 12091 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:40.902726 12091 heartbeater.cc:511] Master 127.3.15.190:39275 requested a full tablet report, sending...
I20260807 20:08:40.903226 11887 ts_manager.cc:194] Registered new tserver with Master: 9818e7e5152e4aa5a6acbe9ad7f08b47 (127.3.15.129:40095)
I20260807 20:08:40.904119 11887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:41077
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:40.928264 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39275
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:41.017055 12096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:41.017256 12096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:41.017278 12096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:41.018384 12096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:41.018416 12096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:41.018445 12096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:41.019799 12096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:41.019999 12096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:41.020174 12096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:41.025170 12104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.025170 12101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.025182 12102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.025599 12096 server_base.cc:1034] running on GCE node
I20260807 20:08:41.025786 12096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:41.025975 12096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:41.027120 12096 hybrid_clock.cc:648] HybridClock initialized: now 1786133321027111 us; error 29 us; skew 500 ppm
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:41.029817 12096 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:41.030315 12096 webserver.cc:460] Webserver started at http://127.3.15.130:45427/ using document root <none> and password file <none>
I20260807 20:08:41.030531 12096 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:41.030575 12096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:41.030664 12096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:41.031735 12096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "7b0c568ee7c44cb28cd66099e7760407"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "ef7e25bb627d13894f87255e613783c9"
server_key_iv: "2da5df0e07a889f5727e31e2d82aae53"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.032048 12096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "7b0c568ee7c44cb28cd66099e7760407"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "ef7e25bb627d13894f87255e613783c9"
server_key_iv: "2da5df0e07a889f5727e31e2d82aae53"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.033401 12096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 20:08:41.034142 12111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.034310 12096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:41.034416 12096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "7b0c568ee7c44cb28cd66099e7760407"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "ef7e25bb627d13894f87255e613783c9"
server_key_iv: "2da5df0e07a889f5727e31e2d82aae53"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.034510 12096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:41.067636 12096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:41.070258 12096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:41.070451 12096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:41.070715 12096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:41.071122 12096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:41.071178 12096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.071223 12096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:41.071249 12096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.077319 12096 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:44733
I20260807 20:08:41.077373 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:44733 every 8 connection(s)
I20260807 20:08:41.077718 12096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:41.084805 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12096
I20260807 20:08:41.084959 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:41.085202 3134 external_mini_cluster.cc:1428] Setting key c5540f91485739a365ad0f744b1da9e3
I20260807 20:08:41.087746 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.078519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:47563 (local address 127.3.15.190:39275)
0807 20:08:41.078602 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.078604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.078805 (+ 201us) server_negotiation.cc:407] Connection header received
0807 20:08:41.079316 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.079317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.079327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.079389 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.080374 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.080890 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.081908 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.082103 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.082156 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.084311 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:41.084313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:41.084313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:41.084321 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:41.086165 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:41.086647 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:41.086648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:41.086649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:41.086722 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:41.086999 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:41.087000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:41.087000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:41.087130 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:41.087204 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:41.087547 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 20:08:41.087652 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 20:08:41.087989 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.078471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:47563)
0807 20:08:41.078671 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.078714 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.079223 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.079460 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.079463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.080056 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.080280 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.080285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.081002 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.081003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.081828 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.081830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.082179 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.082180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.082288 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.082779 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.082788 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.084217 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:41.086252 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:41.086261 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:41.086263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:41.086518 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:41.086803 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:41.086804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:41.086804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:41.086927 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:41.087246 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:41.087253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:41.087401 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 20:08:41.087515 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 20:08:41.087692 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":112,"threads_started":1}
I20260807 20:08:41.088527 12233 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39275
I20260807 20:08:41.088642 12233 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:41.088896 12233 heartbeater.cc:511] Master 127.3.15.190:39275 requested a full tablet report, sending...
I20260807 20:08:41.089345 11887 ts_manager.cc:194] Registered new tserver with Master: 7b0c568ee7c44cb28cd66099e7760407 (127.3.15.130:44733)
I20260807 20:08:41.089907 11887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:47563
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:41.114174 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39275
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:41.200742 12238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:41.201021 12238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:41.201067 12238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:41.202281 12238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:41.202353 12238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:41.202409 12238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:41.203962 12238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:41.204242 12238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:41.204500 12238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:41.209859 12243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.209940 12244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.209931 12246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.210050 12238 server_base.cc:1034] running on GCE node
I20260807 20:08:41.210249 12238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:41.210497 12238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:41.211640 12238 hybrid_clock.cc:648] HybridClock initialized: now 1786133321211630 us; error 31 us; skew 500 ppm
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:41.214293 12238 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:41.214781 12238 webserver.cc:460] Webserver started at http://127.3.15.131:33523/ using document root <none> and password file <none>
I20260807 20:08:41.215013 12238 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:41.215081 12238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:41.215235 12238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:41.216292 12238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "25a41a8d5804496f93e510049a9cb427"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "feed1709747f934e4f53fc3affb5e01e"
server_key_iv: "23d53a51f5a6bec4d1520d831aa44f34"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.216682 12238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "25a41a8d5804496f93e510049a9cb427"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "feed1709747f934e4f53fc3affb5e01e"
server_key_iv: "23d53a51f5a6bec4d1520d831aa44f34"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.218052 12238 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:41.218775 12253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.218971 12238 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.219081 12238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "25a41a8d5804496f93e510049a9cb427"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "feed1709747f934e4f53fc3affb5e01e"
server_key_iv: "23d53a51f5a6bec4d1520d831aa44f34"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.219163 12238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:41.228168 12238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:41.230259 12238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:41.230401 12238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:41.230662 12238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:41.230983 12238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:41.231034 12238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.231074 12238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:41.231110 12238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.236836 12238 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:43993
I20260807 20:08:41.236892 12372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:43993 every 8 connection(s)
I20260807 20:08:41.237267 12238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:41.240000 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12238
I20260807 20:08:41.240108 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:41.240274 3134 external_mini_cluster.cc:1428] Setting key d4c73d235e55b9646579d610d59fca34
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:41.247249 11941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.238106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:47301 (local address 127.3.15.190:39275)
0807 20:08:41.238174 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.238176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.238436 (+ 260us) server_negotiation.cc:407] Connection header received
0807 20:08:41.238994 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.238996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.239010 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.239069 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.240055 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.240771 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.241959 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.242154 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.242213 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.244501 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:41.244503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:41.244503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:41.244510 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:41.245901 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:41.246328 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:41.246330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:41.246330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:41.246404 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:41.246657 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:41.246658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:41.246658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:41.246781 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:41.246852 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:41.247108 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 20:08:41.247182 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:08:41.247401 12375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.238069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:47301)
0807 20:08:41.238273 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.238321 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.238865 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.239144 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.239147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.239742 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.239943 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.239948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.240874 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.240876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.241838 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.241841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.242231 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.242232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.242337 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.242815 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.242824 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.244399 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:41.245989 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:41.245996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:41.245998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:41.246239 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:41.246479 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:41.246479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:41.246479 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:41.246582 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:41.246844 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:41.246848 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:41.246963 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 20:08:41.247047 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 20:08:41.247184 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":118,"threads_started":1}
I20260807 20:08:41.247936 12373 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39275
I20260807 20:08:41.248061 12373 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:41.248250 12373 heartbeater.cc:511] Master 127.3.15.190:39275 requested a full tablet report, sending...
I20260807 20:08:41.248663 11887 ts_manager.cc:194] Registered new tserver with Master: 25a41a8d5804496f93e510049a9cb427 (127.3.15.131:43993)
I20260807 20:08:41.249081 11887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:47301
I20260807 20:08:41.251799 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab.
Entry for principal oryx/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab.
WARNING: no policy specified for oryx/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab.
Entry for principal oryx/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab.
WARNING: no policy specified for oryx/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab.
Entry for principal oryx/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab.
I20260807 20:08:41.332266 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 11954
I20260807 20:08:41.339569 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12096
I20260807 20:08:41.347787 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12238
I20260807 20:08:41.352886 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 11853
I20260807 20:08:41.359577 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39275
--webserver_interface=127.3.15.190
--webserver_port=32861
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:41.443308 12382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:41.443531 12382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:41.443555 12382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:41.444810 12382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:41.444854 12382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:41.444869 12382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:41.444883 12382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:41.444895 12382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:41.446538 12382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=32861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:41.446831 12382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:41.447093 12382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:41.452278 12390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.452288 12387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.452337 12388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.452527 12382 server_base.cc:1034] running on GCE node
I20260807 20:08:41.452814 12382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:41.453207 12382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:41.454361 12382 hybrid_clock.cc:648] HybridClock initialized: now 1786133321454335 us; error 47 us; skew 500 ppm
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:41.457401 12382 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:41.457913 12382 webserver.cc:460] Webserver started at http://127.3.15.190:32861/ using document root <none> and password file <none>
I20260807 20:08:41.458194 12382 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:41.458297 12382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:41.459725 12382 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:41.460585 12397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.460817 12382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:41.460933 12382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "aa08cd3d928a4785ba82358625f0a8b7"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "af830aeb30d040a491111312c9f74ef3"
server_key_iv: "e8203f4dfbc477da1f57e2b1950a66b0"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.461400 12382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:41.474153 12382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:41.476629 12382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:41.476895 12382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:41.481215 12382 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39275
I20260807 20:08:41.481256 12453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39275 every 8 connection(s)
I20260807 20:08:41.481691 12382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:41.483283 12454 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:08:41.483889 12454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Bootstrap starting.
I20260807 20:08:41.485062 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12382
I20260807 20:08:41.485358 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:40095
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:39275
--webserver_port=32871
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:08:41.492853 12454 log.cc:824] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:41.494573 12454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:08:41.494874 12454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Bootstrap complete.
I20260807 20:08:41.497282 12454 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:41.497534 12454 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa08cd3d928a4785ba82358625f0a8b7, State: Initialized, Role: FOLLOWER
I20260807 20:08:41.497717 12454 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:41.497797 12454 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:41.497852 12454 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:41.497926 12454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:08:41.498978 12454 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:41.499100 12454 leader_election.cc:302] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa08cd3d928a4785ba82358625f0a8b7; no voters:
I20260807 20:08:41.499312 12454 leader_election.cc:288] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:08:41.499348 12458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:08:41.499481 12458 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [term 2 LEADER]: Becoming Leader. State: Replica: aa08cd3d928a4785ba82358625f0a8b7, State: Running, Role: LEADER
I20260807 20:08:41.499598 12454 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:41.499570 12458 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } }
I20260807 20:08:41.499933 12460 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa08cd3d928a4785ba82358625f0a8b7. Latest consensus state: current_term: 2 leader_uuid: "aa08cd3d928a4785ba82358625f0a8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } } }
I20260807 20:08:41.499974 12459 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa08cd3d928a4785ba82358625f0a8b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa08cd3d928a4785ba82358625f0a8b7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39275 } } }
I20260807 20:08:41.500113 12459 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:41.500329 12460 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:41.500319 12466 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:41.500948 12466 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:41.501097 12466 catalog_manager.cc:1259] Loaded cluster ID: e58b611d10754cc69675df56fec67030
I20260807 20:08:41.501291 12466 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:41.502465 12466 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:41.502636 12466 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa08cd3d928a4785ba82358625f0a8b7: Loaded TSK: 0
I20260807 20:08:41.502854 12466 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:08:41.576655 12456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:41.576965 12456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:41.577015 12456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:41.578326 12456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:41.578397 12456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:41.578450 12456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:41.580083 12456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.129.keytab
--principal=oryx/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:40095
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=32871
--tserver_master_addrs=127.3.15.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:41.580365 12456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:41.580621 12456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:41.586933 12481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.587016 12482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.587267 12456 server_base.cc:1034] running on GCE node
W20260807 20:08:41.587356 12484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.587570 12456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:41.587827 12456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:41.588999 12456 hybrid_clock.cc:648] HybridClock initialized: now 1786133321588985 us; error 30 us; skew 500 ppm
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:41.591988 12456 init.cc:377] Logged in from keytab as oryx/127.3.15.129@KRBTEST.COM (short username oryx)
I20260807 20:08:41.592550 12456 webserver.cc:460] Webserver started at http://127.3.15.129:32871/ using document root <none> and password file <none>
I20260807 20:08:41.592859 12456 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:41.592952 12456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:41.594777 12456 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:08:41.595470 12491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.595710 12456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:41.595811 12456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "9818e7e5152e4aa5a6acbe9ad7f08b47"
format_stamp: "Formatted at 2026-08-07 20:08:40 on dist-test-slave-1x4g"
server_key: "802d3e685df7e144a403f7a2dd9ffc20"
server_key_iv: "b3cacfd1b00611eb1e8569fe73197a6d"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.596204 12456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:41.610162 12456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:41.612591 12456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:41.612762 12456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:41.613107 12456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:41.613502 12456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:41.613559 12456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.613600 12456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:41.613628 12456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.621290 12456 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40095
I20260807 20:08:41.621372 12610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40095 every 8 connection(s)
I20260807 20:08:41.621778 12456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.630978 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.622524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:36679)
0807 20:08:41.622732 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.622777 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.623396 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.624096 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.624100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.624735 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.625055 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.625061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.626540 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.626542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.627334 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.627336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.627942 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.627943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.628047 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.628543 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.628552 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.630799 (+ 2247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
W20260807 20:08:41.631237 12611 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:39275 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.15.190:39275: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
W20260807 20:08:41.631085 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.622645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:36679 (local address 127.3.15.190:39275)
0807 20:08:41.623612 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.623621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.623658 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:08:41.623805 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.623812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.623880 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.624023 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.625178 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.626432 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.627441 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.627831 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.627936 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.630926 (+ 2990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:36679: BlockingRecv error: recv got EOF from 127.3.15.129:36679 (error 108)
Metrics: {"server-negotiator.queue_time_us":911,"thread_start_us":77,"threads_started":1}
I20260807 20:08:41.631904 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12456
I20260807 20:08:41.632236 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:44733
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:39275
--webserver_port=45427
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.636439 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.631518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:49235)
0807 20:08:41.631553 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.631564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.631605 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.631738 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.631739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.631885 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.631942 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.631944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.632909 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.632910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.633731 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.633734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.634135 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.634136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.634211 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.634693 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.634698 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.636354 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":17}
W20260807 20:08:41.636511 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.631576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:49235 (local address 127.3.15.190:39275)
0807 20:08:41.631612 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.631615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.631642 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:08:41.631660 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.631661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.631670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.631712 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.632046 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.632743 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.633828 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.634025 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.634120 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.636439 (+ 2319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:49235: BlockingRecv error: recv got EOF from 127.3.15.129:49235 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.641084 12613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.636873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:46691)
0807 20:08:41.636915 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.636925 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.636982 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.637135 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.637136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.637264 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.637290 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.637291 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.637906 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.637907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.638402 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.638404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.638774 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.638775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.638829 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.639254 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.639259 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.640992 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:08:41.641148 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.636946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:46691 (local address 127.3.15.190:39275)
0807 20:08:41.637021 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.637023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.637031 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:41.637051 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.637051 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.637063 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.637131 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.637366 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.637823 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.638510 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.638701 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.638776 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.641079 (+ 2303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:46691: BlockingRecv error: recv got EOF from 127.3.15.129:46691 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 20:08:41.641264 12611 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:08:41.717237 12615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:41.717451 12615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:41.717474 12615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:41.718674 12615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:41.718719 12615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:41.718745 12615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:41.720160 12615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.130.keytab
--principal=oryx/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:44733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=45427
--tserver_master_addrs=127.3.15.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:41.720350 12615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:41.720520 12615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:41.725551 12623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.725584 12620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.725666 12621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.725912 12615 server_base.cc:1034] running on GCE node
I20260807 20:08:41.726171 12615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:41.726430 12615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:41.727559 12615 hybrid_clock.cc:648] HybridClock initialized: now 1786133321727536 us; error 54 us; skew 500 ppm
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:41.730350 12615 init.cc:377] Logged in from keytab as oryx/127.3.15.130@KRBTEST.COM (short username oryx)
I20260807 20:08:41.730741 12615 webserver.cc:460] Webserver started at http://127.3.15.130:45427/ using document root <none> and password file <none>
I20260807 20:08:41.730923 12615 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:41.730978 12615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:41.732288 12615 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:41.732916 12630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.733098 12615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:41.733186 12615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "7b0c568ee7c44cb28cd66099e7760407"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "ef7e25bb627d13894f87255e613783c9"
server_key_iv: "2da5df0e07a889f5727e31e2d82aae53"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.733536 12615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:41.741818 12615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:41.744124 12615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:41.744246 12615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:41.744431 12615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:41.744688 12615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:41.744717 12615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.744738 12615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:41.744750 12615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.751811 12615 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:44733
I20260807 20:08:41.751883 12747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:44733 every 8 connection(s)
I20260807 20:08:41.752157 12615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.758421 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12615
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.758756 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:43993
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:39275
--webserver_port=33523
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:08:41.759598 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.752919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:41583)
0807 20:08:41.753086 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.753121 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.753602 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.753811 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.753814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.754404 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.754606 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.754611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.755294 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.755295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.756149 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.756151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.756499 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.756500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.756631 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.757148 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.757156 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.759449 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
W20260807 20:08:41.759642 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.752904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:41583 (local address 127.3.15.190:39275)
0807 20:08:41.752963 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.752965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.753211 (+ 246us) server_negotiation.cc:407] Connection header received
0807 20:08:41.753689 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.753691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.753705 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.753758 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.754690 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.755188 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.756227 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.756407 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.756458 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.759562 (+ 3104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:41583: BlockingRecv error: recv got EOF from 127.3.15.130:41583 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260807 20:08:41.759924 12748 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:39275 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.15.190:39275: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.764432 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.760183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:46025)
0807 20:08:41.760230 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.760240 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.760281 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.760427 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.760427 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.760560 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.760604 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.760605 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.761327 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.761328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.761839 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.761841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.762198 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.762199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.762267 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.762699 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.762704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.764366 (+ 1662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 20:08:41.764586 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.760237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46025 (local address 127.3.15.190:39275)
0807 20:08:41.760297 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.760299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.760308 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:41.760335 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.760336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.760346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.760416 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.760682 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.761245 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.761933 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.762128 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.762202 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.764478 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:46025: BlockingRecv error: recv got EOF from 127.3.15.130:46025 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.769086 12750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.764919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:41817)
0807 20:08:41.764957 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.764969 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.765022 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.765181 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.765182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.765313 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.765339 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.765340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.766007 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.766008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.766499 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.766501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.766852 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.766854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.766906 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.767304 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.767309 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.769003 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 20:08:41.769147 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.765032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:41817 (local address 127.3.15.190:39275)
0807 20:08:41.765075 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.765076 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.765083 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:41.765102 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.765103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.765111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.765178 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.765422 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.765933 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.766584 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.766771 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.766852 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.769079 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:41817: BlockingRecv error: recv got EOF from 127.3.15.130:41817 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:08:41.769486 12748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:08:41.844421 12751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:41.844623 12751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:41.844645 12751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:41.845839 12751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:41.845882 12751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:41.845908 12751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:41.847275 12751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/krb5kdc/oryx_127.3.15.131.keytab
--principal=oryx/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:43993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=33523
--tserver_master_addrs=127.3.15.190:39275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:41.847457 12751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:41.847630 12751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:41.852486 12756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.852481 12759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:41.852572 12757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:41.853107 12751 server_base.cc:1034] running on GCE node
I20260807 20:08:41.853358 12751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:41.853601 12751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:41.854779 12751 hybrid_clock.cc:648] HybridClock initialized: now 1786133321854758 us; error 38 us; skew 500 ppm
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133321, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:41.862339 12751 init.cc:377] Logged in from keytab as oryx/127.3.15.131@KRBTEST.COM (short username oryx)
I20260807 20:08:41.862900 12751 webserver.cc:460] Webserver started at http://127.3.15.131:33523/ using document root <none> and password file <none>
I20260807 20:08:41.863157 12751 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:41.863257 12751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:41.864912 12751 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:08:41.865599 12766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.865775 12751 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:41.865887 12751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "25a41a8d5804496f93e510049a9cb427"
format_stamp: "Formatted at 2026-08-07 20:08:41 on dist-test-slave-1x4g"
server_key: "feed1709747f934e4f53fc3affb5e01e"
server_key_iv: "23d53a51f5a6bec4d1520d831aa44f34"
server_key_version: "encryptionkey@0"
I20260807 20:08:41.866266 12751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:41.880908 12751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:41.882789 12751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:41.882956 12751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:41.883203 12751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:41.883519 12751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:41.883574 12751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.883610 12751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:41.883659 12751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:41.889523 12751 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:43993
I20260807 20:08:41.889577 12886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:43993 every 8 connection(s)
I20260807 20:08:41.889992 12751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:41.894920 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12751
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.897702 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.890728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:51541)
0807 20:08:41.890872 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.890908 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.891389 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.891607 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.891610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.892176 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.892368 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.892373 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.893081 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.893082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.893902 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.893904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.894231 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.894232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.894333 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.894836 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.894845 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.897559 (+ 2714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
W20260807 20:08:41.897740 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.890741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:51541 (local address 127.3.15.190:39275)
0807 20:08:41.890792 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.890795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.891005 (+ 210us) server_negotiation.cc:407] Connection header received
0807 20:08:41.891484 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.891486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.891496 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.891545 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.892443 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.893003 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.893977 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.894161 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.894211 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.897677 (+ 3466us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:51541: BlockingRecv error: recv got EOF from 127.3.15.131:51541 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 20:08:41.898041 12887 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:39275 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.15.190:39275: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.902982 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.898274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:47595)
0807 20:08:41.898314 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.898324 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.898355 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.898523 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.898524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.898657 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.898700 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.898701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.899600 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.899601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.900349 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.900352 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.900749 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.900750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.900854 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.901161 (+ 307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.901167 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.902915 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 20:08:41.903051 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.898354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:47595 (local address 127.3.15.190:39275)
0807 20:08:41.898387 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.898389 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.898404 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:08:41.898421 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.898422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.898429 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.898505 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.898774 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.899515 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.900422 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.900674 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.900727 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.902984 (+ 2257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:47595: BlockingRecv error: recv got EOF from 127.3.15.131:47595 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:08:41.903673 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.895451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.3.15.190:39275)
0807 20:08:41.895635 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.895637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.895645 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:41.895668 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.895668 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.895679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.895750 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.896506 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.897155 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.898109 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.898461 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.898536 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.899985 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:41.899990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:41.899990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:41.900002 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:41.901941 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:41.902311 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:41.902315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:41.902315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:41.902417 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:41.902654 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:41.902656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:41.902656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:41.902977 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:41.903275 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:41.903393 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 20:08:41.903512 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:41 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:41.907553 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:41.903275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:60537)
0807 20:08:41.903345 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:41.903355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:41.903387 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:41.903554 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:41.903555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:41.903682 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:41.903706 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.903707 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.904568 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.904570 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.905200 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:41.905203 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.905610 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.905611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:41.905665 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.906056 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:41.906061 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:41.907511 (+ 1450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 20:08:41.907639 12614 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:41.903407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:60537 (local address 127.3.15.190:39275)
0807 20:08:41.903444 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:41.903445 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:41.903452 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:08:41.903478 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:41.903479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:41.903486 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:41.903550 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:41.903962 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.904497 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.905281 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:41.905523 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:41.905604 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:41.907586 (+ 1982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:60537: BlockingRecv error: recv got EOF from 127.3.15.131:60537 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 20:08:41.907717 12887 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 20:08:42 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:42 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:42.647099 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:42.641752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:38103)
0807 20:08:42.642023 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:42.642040 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:42.642118 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:42.642328 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:42.642329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:42.642588 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:42.642663 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:42.642666 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.643546 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.643547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:42.644106 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:42.644108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.644468 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.644469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:42.644516 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:42.644989 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:42.644994 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:42.647029 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":78,"threads_started":1}
W20260807 20:08:42.647266 12893 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:42.641991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:38103 (local address 127.3.15.190:39275)
0807 20:08:42.642175 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:42.642178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:42.642189 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:42.642223 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:42.642224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:42.642234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:42.642293 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:42.642756 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.643458 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.644187 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.644385 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.644441 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:42.647155 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:38103: BlockingRecv error: recv got EOF from 127.3.15.129:38103 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":106,"threads_started":1}
Aug 07 20:08:42 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:42 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:42.775274 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:42.769980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:59957)
0807 20:08:42.770168 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:42.770188 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:42.770254 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:42.770417 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:42.770418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:42.770600 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:42.770652 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:42.770654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.771407 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.771408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:42.772039 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:42.772042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.772409 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.772410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:42.772483 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:42.773135 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:42.773141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:42.775194 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
W20260807 20:08:42.775468 12893 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:42.770095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:59957 (local address 127.3.15.190:39275)
0807 20:08:42.770208 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:42.770211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:42.770236 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:08:42.770317 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:42.770318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:42.770330 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:42.770382 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:42.770752 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.771312 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.772132 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.772334 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.772395 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:42.775378 (+ 2983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:59957: BlockingRecv error: recv got EOF from 127.3.15.130:59957 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 20:08:42 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:42 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:42.913525 12895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:42.908175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:42863)
0807 20:08:42.908381 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:42.908397 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:42.908502 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:42.908692 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:42.908693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:42.908938 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:42.909024 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:42.909026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.909771 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.909772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:42.910362 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:42.910363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.910802 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.910804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:42.910897 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:42.911537 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:42.911544 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:42.913416 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
W20260807 20:08:42.913589 12893 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:42.908322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:42863 (local address 127.3.15.190:39275)
0807 20:08:42.908402 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:42.908404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:42.908489 (+ 85us) server_negotiation.cc:407] Connection header received
0807 20:08:42.908594 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:42.908596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:42.908612 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:42.908684 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:42.909114 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.909679 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.910448 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:42.910679 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:42.910749 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:42.913514 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:42863: BlockingRecv error: recv got EOF from 127.3.15.131:42863 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 20:08:43 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:43 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:43.652522 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:43.647658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:52659)
0807 20:08:43.647827 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:43.647842 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:43.647907 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:43.648106 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:43.648107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:43.648322 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:43.648400 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:43.648402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.649108 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.649110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:43.649644 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:43.649646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.649999 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.650000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:43.650058 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:43.650524 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:43.650529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:43.652444 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":77,"threads_started":1}
W20260807 20:08:43.652621 12898 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:43.647733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:52659 (local address 127.3.15.190:39275)
0807 20:08:43.647895 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:43.647897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:43.647907 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:08:43.647973 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:43.647974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:43.647986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:43.648080 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:43.648478 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.649016 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.649721 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.649908 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.649968 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:43.652540 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:52659: BlockingRecv error: recv got EOF from 127.3.15.129:52659 (error 108)
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
Aug 07 20:08:43 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:43 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:43.780543 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:43.775763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:36933)
0807 20:08:43.775906 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:43.775916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:43.775972 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:43.776106 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:43.776107 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:43.776272 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:43.776324 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:43.776326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.777048 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.777049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:43.777579 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:43.777581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.777966 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.777967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:43.778036 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:43.778528 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:43.778533 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:43.780440 (+ 1907us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
W20260807 20:08:43.780633 12898 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:43.775848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:36933 (local address 127.3.15.190:39275)
0807 20:08:43.775903 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:43.775906 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:43.775970 (+ 64us) server_negotiation.cc:407] Connection header received
0807 20:08:43.776018 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:43.776019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:43.776032 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:43.776086 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:43.776395 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.776963 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.777685 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.777888 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.777966 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:43.780562 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:36933: BlockingRecv error: recv got EOF from 127.3.15.130:36933 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 20:08:43 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:43 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:43.919193 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:43.914146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:46055)
0807 20:08:43.914315 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:43.914337 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:43.914389 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:43.914573 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:43.914574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:43.914757 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:43.914812 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:43.914814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.915475 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.915476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:43.916054 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:43.916055 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.916386 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.916387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:43.916435 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:43.916982 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:43.916987 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:43.919069 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
W20260807 20:08:43.919570 12898 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:43.914280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:46055 (local address 127.3.15.190:39275)
0807 20:08:43.914371 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:43.914373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:43.914384 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:08:43.914457 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:43.914457 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:43.914472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:43.914530 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:43.914886 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.915401 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.916135 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:43.916314 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:43.916366 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:43.919204 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:46055: BlockingRecv error: recv got EOF from 127.3.15.131:46055 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 20:08:44 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:44 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:44.658147 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:44.653170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:57495)
0807 20:08:44.653404 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:44.653417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:44.653521 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:44.653692 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:44.653693 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:44.653980 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:44.654044 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:44.654046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.654725 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.654726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:44.655293 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:44.655295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.655657 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.655658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:44.655707 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:44.656146 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:44.656151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:44.658088 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":177,"threads_started":1}
W20260807 20:08:44.658278 12902 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:44.653212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:57495 (local address 127.3.15.190:39275)
0807 20:08:44.653428 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:44.653431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:44.653540 (+ 109us) server_negotiation.cc:407] Connection header received
0807 20:08:44.653573 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:44.653574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:44.653590 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:44.653664 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:44.654124 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.654650 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.655378 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.655587 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.655643 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:44.658191 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:57495: BlockingRecv error: recv got EOF from 127.3.15.129:57495 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
Aug 07 20:08:44 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:44 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:44.786113 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:44.781106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:39309)
0807 20:08:44.781289 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:44.781301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:44.781538 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:44.781715 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:44.781716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:44.781909 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:44.781962 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:44.781964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.782640 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.782641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:44.783165 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:44.783166 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.783534 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.783535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:44.783596 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:44.784108 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:44.784114 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:44.786002 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
W20260807 20:08:44.786158 12902 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:44.781199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:39309 (local address 127.3.15.190:39275)
0807 20:08:44.781275 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:44.781279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:44.781510 (+ 231us) server_negotiation.cc:407] Connection header received
0807 20:08:44.781615 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:44.781616 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:44.781636 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:44.781702 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:44.782046 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.782545 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.783246 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.783444 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.783532 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:44.786094 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:39309: BlockingRecv error: recv got EOF from 127.3.15.130:39309 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 20:08:44 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:44 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:44.924443 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:44.919764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:34821)
0807 20:08:44.919916 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:44.919926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:44.919985 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:44.920118 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:44.920119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:44.920291 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:44.920343 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:44.920344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.921091 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.921092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:44.921631 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:44.921633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.921943 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.921944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:44.921984 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:44.922420 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:44.922424 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:44.924381 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260807 20:08:44.924598 12902 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:44.919862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:34821 (local address 127.3.15.190:39275)
0807 20:08:44.919924 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:44.919926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:44.919972 (+ 46us) server_negotiation.cc:407] Connection header received
0807 20:08:44.920031 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:44.920031 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:44.920043 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:44.920096 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:44.920406 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.921023 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.921704 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:44.921884 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:44.921960 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:44.924530 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:34821: BlockingRecv error: recv got EOF from 127.3.15.131:34821 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 07 20:08:45 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:45 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:45.663290 12905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:45.658666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:45507)
0807 20:08:45.658817 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:45.658828 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:45.658888 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:45.659031 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:45.659032 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:45.659203 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:45.659254 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:45.659256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.659909 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.659910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:45.660475 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:45.660477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.660861 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.660862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:45.660913 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:45.661500 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:45.661505 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:45.663230 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":88,"threads_started":1}
W20260807 20:08:45.663466 12906 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:45.658711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:45507 (local address 127.3.15.190:39275)
0807 20:08:45.658836 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:45.658838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:45.658870 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:08:45.658939 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:45.658940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:45.658950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:45.659012 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:45.659320 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.659831 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.660543 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.660750 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.660864 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:45.663382 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:45507: BlockingRecv error: recv got EOF from 127.3.15.129:45507 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
Aug 07 20:08:45 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:45 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:45.792006 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:45.786623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:56107)
0807 20:08:45.786821 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:45.786831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:45.786888 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:45.787115 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:45.787116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:45.787300 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:45.787354 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:45.787356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.788087 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.788088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:45.788760 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:45.788765 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.789379 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.789380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:45.789453 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:45.790033 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:45.790039 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:45.791941 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":113,"threads_started":1}
W20260807 20:08:45.792165 12906 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:45.786721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56107 (local address 127.3.15.190:39275)
0807 20:08:45.786807 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:45.786811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:45.786899 (+ 88us) server_negotiation.cc:407] Connection header received
0807 20:08:45.786961 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:45.786961 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:45.786976 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:45.787044 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:45.787454 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.787996 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.788909 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.789295 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.789370 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:45.792073 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:56107: BlockingRecv error: recv got EOF from 127.3.15.130:56107 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 20:08:45 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:45 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:45.929432 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:45.925033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:34883)
0807 20:08:45.925180 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:45.925197 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:45.925258 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:45.925396 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:45.925397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:45.925567 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:45.925622 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:45.925624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.926270 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.926271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:45.926788 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:45.926789 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.927145 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.927146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:45.927188 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:45.927640 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:45.927644 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:45.929372 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
W20260807 20:08:45.929543 12906 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:45.925118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:34883 (local address 127.3.15.190:39275)
0807 20:08:45.925168 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:45.925170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:45.925260 (+ 90us) server_negotiation.cc:407] Connection header received
0807 20:08:45.925305 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:45.925306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:45.925321 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:45.925375 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:45.925691 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.926199 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.926865 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:45.927053 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:45.927137 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:45.929466 (+ 2329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:34883: BlockingRecv error: recv got EOF from 127.3.15.131:34883 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 07 20:08:46 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:46 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:46.669070 12910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:46.663839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:36831)
0807 20:08:46.664015 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:46.664026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:46.664082 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:46.664363 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:46.664363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:46.664575 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:46.664634 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:46.664636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.665638 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.665638 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:46.666169 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:46.666171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.666589 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.666590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:46.666653 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:46.667112 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:46.667117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:46.669010 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
W20260807 20:08:46.669239 12911 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:46.664038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:36831 (local address 127.3.15.190:39275)
0807 20:08:46.664201 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:46.664203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:46.664229 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:08:46.664259 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:46.664260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:46.664272 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:46.664343 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:46.664716 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.665546 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.666286 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.666514 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.666579 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:46.669133 (+ 2554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:36831: BlockingRecv error: recv got EOF from 127.3.15.129:36831 (error 108)
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":104,"threads_started":1}
Aug 07 20:08:46 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:46 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:46.797343 12912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:46.792524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:56153)
0807 20:08:46.792670 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:46.792682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:46.792765 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:46.792958 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:46.792959 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:46.793168 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:46.793217 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:46.793219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.793933 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.793934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:46.794456 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:46.794458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.794823 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.794824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:46.794890 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:46.795336 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:46.795341 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:46.797242 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
W20260807 20:08:46.797398 12911 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:46.792701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56153 (local address 127.3.15.190:39275)
0807 20:08:46.792767 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:46.792769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:46.792803 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:08:46.792871 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:46.792872 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:46.792887 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:46.792952 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:46.793310 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.793833 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.794545 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.794746 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.794826 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:46.797332 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:56153: BlockingRecv error: recv got EOF from 127.3.15.130:56153 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 20:08:46 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:46 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:46.938998 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:46.929981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:54509)
0807 20:08:46.930143 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:46.930157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:46.930224 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:46.930364 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:46.930365 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:46.930541 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:46.930602 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:46.930604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.931269 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.931270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:46.931823 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:46.931825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.932182 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.932183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:46.932224 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:46.937048 (+ 4824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:46.937053 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:46.938936 (+ 1883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":102,"threads_started":1}
W20260807 20:08:46.939148 12911 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:46.930067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:54509 (local address 127.3.15.190:39275)
0807 20:08:46.930135 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:46.930142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:46.930222 (+ 80us) server_negotiation.cc:407] Connection header received
0807 20:08:46.930271 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:46.930271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:46.930284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:46.930341 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:46.930666 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.931202 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.931913 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:46.932111 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:46.932169 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:46.939059 (+ 6890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:54509: BlockingRecv error: recv got EOF from 127.3.15.131:54509 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 20:08:47 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:47 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:47.674438 12914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:47.669562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:42991)
0807 20:08:47.669708 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:47.669720 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:47.669829 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:47.670013 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:47.670014 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:47.670204 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:47.670254 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:47.670256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.670935 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.670936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:47.671496 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:47.671498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.671847 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.671848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:47.671900 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:47.672366 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:47.672372 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:47.674373 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":118,"threads_started":1}
W20260807 20:08:47.674547 12915 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:47.669630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:42991 (local address 127.3.15.190:39275)
0807 20:08:47.669763 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:47.669766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:47.669778 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:08:47.669907 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:47.669908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:47.669922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:47.669993 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:47.670314 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.670839 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.671570 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.671769 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.671821 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:47.674469 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:42991: BlockingRecv error: recv got EOF from 127.3.15.129:42991 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
Aug 07 20:08:47 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:47 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:47.802875 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:47.797812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:50779)
0807 20:08:47.797939 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:47.797953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:47.798024 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:47.798188 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:47.798189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:47.798397 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:47.798446 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:47.798448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.799238 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.799239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:47.799980 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:47.799983 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.800339 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.800340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:47.800399 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:47.801002 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:47.801007 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:47.802817 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
W20260807 20:08:47.803067 12915 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:47.797928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:50779 (local address 127.3.15.190:39275)
0807 20:08:47.797977 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:47.797979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:47.798004 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:08:47.798089 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:47.798091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:47.798109 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:47.798166 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:47.798527 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.799154 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.800063 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.800265 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.800341 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:47.802992 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:50779: BlockingRecv error: recv got EOF from 127.3.15.130:50779 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 07 20:08:47 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:47 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:47.945048 12917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:47.939466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:58683)
0807 20:08:47.939685 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:47.939707 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:47.939812 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:47.940037 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:47.940038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:47.940264 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:47.940321 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:47.940323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.941228 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.941230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:47.941840 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:47.941842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.942300 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.942301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:47.942386 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:47.942899 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:47.942906 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:47.944980 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":107,"threads_started":1}
W20260807 20:08:47.945133 12915 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:47.939572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:58683 (local address 127.3.15.190:39275)
0807 20:08:47.939666 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:47.939669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:47.939785 (+ 116us) server_negotiation.cc:407] Connection header received
0807 20:08:47.939905 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:47.939907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:47.939925 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:47.940024 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:47.940427 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.941118 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.941965 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:47.942193 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:47.942297 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:47.945080 (+ 2783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:58683: BlockingRecv error: recv got EOF from 127.3.15.131:58683 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 20:08:48 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:48 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:48.679514 12918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:48.674945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:56165)
0807 20:08:48.675155 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:48.675170 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:48.675241 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:48.675365 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:48.675365 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:48.675541 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:48.675594 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:48.675595 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.676227 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.676227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:48.676761 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:48.676762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.677143 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.677144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:48.677194 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:48.677647 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:48.677652 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:48.679454 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":64,"threads_started":1}
W20260807 20:08:48.679672 12919 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:48.675087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:56165 (local address 127.3.15.190:39275)
0807 20:08:48.675222 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:48.675224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:48.675233 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:48.675299 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:48.675299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:48.675310 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:48.675358 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:48.675658 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.676159 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.676861 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.677071 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.677125 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:48.679572 (+ 2447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:56165: BlockingRecv error: recv got EOF from 127.3.15.129:56165 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
Aug 07 20:08:48 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:48 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:48.808337 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:48.803490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:42385)
0807 20:08:48.803682 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:48.803693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:48.803753 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:48.803885 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:48.803885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:48.804075 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:48.804132 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:48.804134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.804859 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.804861 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:48.805496 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:48.805499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.805843 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.805844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:48.805887 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:48.806404 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:48.806410 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:48.808264 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
W20260807 20:08:48.808488 12919 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:48.803569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:42385 (local address 127.3.15.190:39275)
0807 20:08:48.803641 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:48.803643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:48.803748 (+ 105us) server_negotiation.cc:407] Connection header received
0807 20:08:48.803799 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:48.803800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:48.803811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:48.803864 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:48.804197 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.804728 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.805578 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.805772 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.805829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:48.808387 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:42385: BlockingRecv error: recv got EOF from 127.3.15.130:42385 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 20:08:48 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:48 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:48.950131 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:48.945567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:42609)
0807 20:08:48.945731 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:48.945741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:48.945803 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:48.946052 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:48.946053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:48.946237 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:48.946289 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:48.946291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.946956 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.946956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:48.947489 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:48.947491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.947813 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.947814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:48.947855 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:48.948284 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:48.948289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:48.950071 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":106,"threads_started":1}
W20260807 20:08:48.950266 12919 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:48.945649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:42609 (local address 127.3.15.190:39275)
0807 20:08:48.945730 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:48.945733 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:48.945801 (+ 68us) server_negotiation.cc:407] Connection header received
0807 20:08:48.945847 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:48.945848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:48.945861 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:48.945957 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:48.946379 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.946883 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.947568 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:48.947750 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:48.947800 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:48.950180 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:42609: BlockingRecv error: recv got EOF from 127.3.15.131:42609 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 20:08:49 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:49 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:49.684823 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:49.679964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:45761)
0807 20:08:49.680105 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:49.680116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:49.680166 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:49.680435 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:49.680436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:49.680626 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:49.680678 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:49.680679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.681415 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.681416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:49.681954 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:49.681956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.682309 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.682309 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:49.682353 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:49.682869 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:49.682874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:49.684706 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
W20260807 20:08:49.684983 12923 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:49.680124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:45761 (local address 127.3.15.190:39275)
0807 20:08:49.680283 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:49.680285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:49.680293 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:08:49.680324 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:49.680325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:49.680336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:49.680423 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:49.680750 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.681332 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.682035 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.682236 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.682297 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:49.684892 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:45761: BlockingRecv error: recv got EOF from 127.3.15.129:45761 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
Aug 07 20:08:49 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:49 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:49.813776 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:49.808885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:39371)
0807 20:08:49.809005 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:49.809017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:49.809083 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:49.809240 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:49.809240 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:49.809406 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:49.809456 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:49.809457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.810226 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.810227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:49.810748 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:49.810750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.811127 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.811129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:49.811184 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:49.811752 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:49.811757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:49.813705 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
W20260807 20:08:49.813849 12923 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:49.808908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:39371 (local address 127.3.15.190:39275)
0807 20:08:49.808954 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:49.808957 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:49.809085 (+ 128us) server_negotiation.cc:407] Connection header received
0807 20:08:49.809165 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:49.809165 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:49.809177 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:49.809218 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:49.809514 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.810156 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.810824 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.811027 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.811084 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:49.813799 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:39371: BlockingRecv error: recv got EOF from 127.3.15.130:39371 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 20:08:49 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:49 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:49.955606 12925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:49.950679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:51623)
0807 20:08:49.950882 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:49.950893 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:49.950994 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:49.951151 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:49.951152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:49.951348 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:49.951398 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:49.951400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.952112 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.952114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:49.952664 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:49.952666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.953055 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.953056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:49.953117 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:49.953597 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:49.953602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:49.955491 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":127,"threads_started":1}
W20260807 20:08:49.955651 12923 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:49.950775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:51623 (local address 127.3.15.190:39275)
0807 20:08:49.950847 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:49.950850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:49.950970 (+ 120us) server_negotiation.cc:407] Connection header received
0807 20:08:49.951053 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:49.951054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:49.951069 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:49.951145 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:49.951503 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.952025 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.952754 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:49.952978 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:49.953056 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:49.955574 (+ 2518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:51623: BlockingRecv error: recv got EOF from 127.3.15.131:51623 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 20:08:50 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:50 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:50.690068 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:50.685395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:34637)
0807 20:08:50.685551 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:50.685566 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:50.685641 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:50.685782 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:50.685783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:50.685953 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:50.686004 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:50.686006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.686685 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.686686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:50.687270 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:50.687272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.687616 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.687617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:50.687657 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:50.688128 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:50.688133 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:50.690011 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
W20260807 20:08:50.690237 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:50.685441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:34637 (local address 127.3.15.190:39275)
0807 20:08:50.685593 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:50.685595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:50.685621 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:08:50.685693 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:50.685693 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:50.685705 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:50.685766 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:50.686068 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.686619 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.687344 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.687549 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.687604 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:50.690120 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:34637: BlockingRecv error: recv got EOF from 127.3.15.129:34637 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
Aug 07 20:08:50 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:50 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:50.819201 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:50.814268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:53477)
0807 20:08:50.814442 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:50.814456 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:50.814550 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:50.814733 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:50.814734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:50.814971 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:50.815022 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:50.815024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.815727 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.815728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:50.816269 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:50.816271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.816683 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.816684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:50.816754 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:50.817279 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:50.817284 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:50.819126 (+ 1842us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
W20260807 20:08:50.819299 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:50.814370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:53477 (local address 127.3.15.190:39275)
0807 20:08:50.814450 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:50.814453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:50.814528 (+ 75us) server_negotiation.cc:407] Connection header received
0807 20:08:50.814634 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:50.814635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:50.814651 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:50.814722 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:50.815120 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.815636 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.816358 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.816567 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.816684 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:50.819230 (+ 2546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:53477: BlockingRecv error: recv got EOF from 127.3.15.130:53477 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 20:08:50 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:50 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:50.960891 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:50.956086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:33935)
0807 20:08:50.956273 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:50.956291 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:50.956348 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:50.956480 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:50.956480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:50.956680 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:50.956741 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:50.956743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.957529 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.957531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:50.958084 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:50.958086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.958458 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.958458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:50.958500 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:50.958953 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:50.958957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:50.960819 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":116,"threads_started":1}
W20260807 20:08:50.960973 12927 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:50.956172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:33935 (local address 127.3.15.190:39275)
0807 20:08:50.956221 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:50.956223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:50.956347 (+ 124us) server_negotiation.cc:407] Connection header received
0807 20:08:50.956394 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:50.956395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:50.956407 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:50.956471 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:50.956851 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.957434 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.958187 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:50.958384 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:50.958456 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:50.960889 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:33935: BlockingRecv error: recv got EOF from 127.3.15.131:33935 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 07 20:08:51 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:51 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:51.695370 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:51.690622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:57893)
0807 20:08:51.690783 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:51.690798 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:51.690862 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:51.691000 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:51.691001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:51.691172 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:51.691233 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:51.691235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.691894 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.691894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:51.692418 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:51.692419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.692757 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.692758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:51.692861 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:51.693322 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:51.693327 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:51.695311 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
W20260807 20:08:51.695531 12931 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:51.690684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:57893 (local address 127.3.15.190:39275)
0807 20:08:51.690853 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:51.690855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:51.690864 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:51.690918 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:51.690919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:51.690929 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:51.690979 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:51.691298 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.691826 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.692489 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.692693 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.692744 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:51.695423 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:57893: BlockingRecv error: recv got EOF from 127.3.15.129:57893 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":105,"threads_started":1}
Aug 07 20:08:51 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:51 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:51.824127 12932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:51.819760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:41819)
0807 20:08:51.819943 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:51.819954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:51.820018 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:51.820140 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:51.820140 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:51.820309 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:51.820359 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:51.820361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.821013 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.821014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:51.821537 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:51.821538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.821870 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.821871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:51.821914 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:51.822340 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:51.822345 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:51.824066 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":111,"threads_started":1}
W20260807 20:08:51.824242 12931 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:51.819819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:41819 (local address 127.3.15.190:39275)
0807 20:08:51.819886 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:51.819889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:51.820017 (+ 128us) server_negotiation.cc:407] Connection header received
0807 20:08:51.820068 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:51.820069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:51.820081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:51.820126 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:51.820412 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.820943 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.821608 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.821803 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.821862 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:51.824161 (+ 2299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:41819: BlockingRecv error: recv got EOF from 127.3.15.130:41819 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 20:08:51 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:51 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:51.970945 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:51.961336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:47851)
0807 20:08:51.961554 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:51.961602 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:51.961716 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:51.961969 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:51.961970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:51.962169 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:51.962220 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:51.962222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.962936 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.962937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:51.963494 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:51.963496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.963867 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.963868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:51.963929 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:51.968863 (+ 4934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:51.968870 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:51.970879 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":102,"threads_started":1}
W20260807 20:08:51.971107 12931 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:51.961474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:47851 (local address 127.3.15.190:39275)
0807 20:08:51.961550 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:51.961553 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:51.961674 (+ 121us) server_negotiation.cc:407] Connection header received
0807 20:08:51.961811 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:51.961813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:51.961828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:51.961954 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:51.962302 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.962811 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.963588 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:51.963789 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:51.963868 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:51.971026 (+ 7158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:47851: BlockingRecv error: recv got EOF from 127.3.15.131:47851 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 07 20:08:52 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:52 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:52.701048 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:52.695820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:45989)
0807 20:08:52.695954 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:52.695974 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:52.696027 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:52.696222 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:52.696222 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:52.696399 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:52.696453 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:52.696454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.697186 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.697187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:52.697747 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:52.697748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.698137 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.698138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:52.698184 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:52.698675 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:52.698682 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:52.700979 (+ 2297us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
W20260807 20:08:52.701193 12935 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:52.695920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:45989 (local address 127.3.15.190:39275)
0807 20:08:52.696086 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:52.696088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:52.696097 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:52.696131 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:52.696131 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:52.696142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:52.696205 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:52.696523 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.697108 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.697825 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.698064 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.698123 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:52.701078 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:45989: BlockingRecv error: recv got EOF from 127.3.15.129:45989 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":112,"threads_started":1}
Aug 07 20:08:52 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:52 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:52.829825 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:52.824758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:41421)
0807 20:08:52.824963 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:52.824974 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:52.825040 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:52.825191 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:52.825191 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:52.825372 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:52.825422 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:52.825424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.826147 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.826148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:52.826707 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:52.826709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.827198 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.827199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:52.827273 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:52.827812 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:52.827818 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:52.829717 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":138,"threads_started":1}
W20260807 20:08:52.829860 12935 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:52.824846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:41421 (local address 127.3.15.190:39275)
0807 20:08:52.824918 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:52.824920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:52.825062 (+ 142us) server_negotiation.cc:407] Connection header received
0807 20:08:52.825098 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:52.825099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:52.825113 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:52.825173 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:52.825498 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.826039 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.826804 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.827100 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.827200 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:52.829794 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:41421: BlockingRecv error: recv got EOF from 127.3.15.130:41421 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 20:08:52 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:52 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:52.976372 12937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:52.971402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:58165)
0807 20:08:52.971696 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:52.971708 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:52.971811 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:52.971983 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:52.971984 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:52.972303 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:52.972369 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:52.972380 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.973109 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.973110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:52.973661 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:52.973663 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.973980 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.973981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:52.974023 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:52.974468 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:52.974473 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:52.976295 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":107,"threads_started":1}
W20260807 20:08:52.976498 12935 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:52.971509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:58165 (local address 127.3.15.190:39275)
0807 20:08:52.971694 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:52.971696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:52.971792 (+ 96us) server_negotiation.cc:407] Connection header received
0807 20:08:52.971895 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:52.971896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:52.971911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:52.971971 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:52.972481 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.973042 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.973737 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:52.973918 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:52.973970 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:52.976395 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:58165: BlockingRecv error: recv got EOF from 127.3.15.131:58165 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Aug 07 20:08:53 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:53 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:53.706804 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:53.701740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:38051)
0807 20:08:53.701958 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:53.701971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:53.702048 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:53.702265 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:53.702266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:53.702499 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:53.702552 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:53.702554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.703224 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.703225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:53.703790 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:53.703792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.704134 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.704134 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:53.704183 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:53.704670 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:53.704676 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:53.706703 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
W20260807 20:08:53.706871 12939 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:53.701790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:38051 (local address 127.3.15.190:39275)
0807 20:08:53.701956 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:53.701958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:53.702035 (+ 77us) server_negotiation.cc:407] Connection header received
0807 20:08:53.702153 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:53.702154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:53.702169 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:53.702236 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:53.702638 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.703149 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.703868 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.704063 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.704118 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:53.706796 (+ 2678us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:38051: BlockingRecv error: recv got EOF from 127.3.15.129:38051 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
Aug 07 20:08:53 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:53 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:53.841101 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:53.836314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:44723)
0807 20:08:53.836475 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:53.836488 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:53.836554 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:53.836706 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:53.836707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:53.836905 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:53.836989 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:53.836991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.837747 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.837748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:53.838323 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:53.838325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.838668 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.838669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:53.838711 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:53.839189 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:53.839194 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:53.841032 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
W20260807 20:08:53.841266 12939 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:53.836380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:44723 (local address 127.3.15.190:39275)
0807 20:08:53.836470 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:53.836472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:53.836554 (+ 82us) server_negotiation.cc:407] Connection header received
0807 20:08:53.836603 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:53.836603 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:53.836627 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:53.836687 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:53.837074 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.837665 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.838414 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.838601 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.838656 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:53.841169 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:44723: BlockingRecv error: recv got EOF from 127.3.15.130:44723 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 20:08:53 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:53 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:53.981479 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:53.976945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:40459)
0807 20:08:53.977109 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:53.977120 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:53.977178 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:53.977296 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:53.977296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:53.977487 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:53.977557 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:53.977558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.978189 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.978190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:53.978726 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:53.978727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.979044 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.979045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:53.979086 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:53.979517 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:53.979521 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:53.981411 (+ 1890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
W20260807 20:08:53.981560 12939 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:53.977004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:40459 (local address 127.3.15.190:39275)
0807 20:08:53.977071 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:53.977073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:53.977179 (+ 106us) server_negotiation.cc:407] Connection header received
0807 20:08:53.977224 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:53.977225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:53.977238 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:53.977289 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:53.977621 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.978118 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.978797 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:53.978979 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:53.979029 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:53.981509 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:40459: BlockingRecv error: recv got EOF from 127.3.15.131:40459 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 20:08:54 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:54 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:54.712383 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:54.707365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:56299)
0807 20:08:54.707524 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:54.707535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:54.707593 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:54.707742 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:54.707742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:54.707933 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:54.707989 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:54.707991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.708646 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.708648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:54.709300 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:54.709303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.709675 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.709676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:54.709730 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:54.710201 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:54.710206 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:54.712283 (+ 2077us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260807 20:08:54.712448 12943 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:54.707423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:56299 (local address 127.3.15.190:39275)
0807 20:08:54.707563 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:54.707565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:54.707578 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:08:54.707656 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:54.707657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:54.707669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:54.707721 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:54.708059 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.708565 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.709380 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.709576 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.709642 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:54.712373 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:56299: BlockingRecv error: recv got EOF from 127.3.15.129:56299 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
Aug 07 20:08:54 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:54 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:54.846365 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:54.841592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:56059)
0807 20:08:54.841775 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:54.841789 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:54.841865 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:54.842061 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:54.842063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:54.842262 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:54.842308 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:54.842309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.842996 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.842998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:54.843550 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:54.843551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.843925 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.843926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:54.843987 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:54.844490 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:54.844495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:54.846293 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
W20260807 20:08:54.846524 12943 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:54.841713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56059 (local address 127.3.15.190:39275)
0807 20:08:54.841779 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:54.841782 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:54.841849 (+ 67us) server_negotiation.cc:407] Connection header received
0807 20:08:54.841975 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:54.841976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:54.841988 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:54.842052 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:54.842391 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.842904 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.843638 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.843839 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.843924 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:54.846429 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:56059: BlockingRecv error: recv got EOF from 127.3.15.130:56059 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 07 20:08:54 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:54 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:54.986477 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:54.981961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:43237)
0807 20:08:54.982079 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:54.982096 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:54.982150 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:54.982267 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:54.982267 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:54.982444 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:54.982494 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:54.982495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.983139 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.983140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:54.983699 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:54.983701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.984017 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.984017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:54.984063 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:54.984505 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:54.984511 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:54.986410 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":58,"threads_started":1}
W20260807 20:08:54.986572 12943 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:54.982032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:43237 (local address 127.3.15.190:39275)
0807 20:08:54.982129 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:54.982131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:54.982140 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:54.982208 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:54.982208 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:54.982221 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:54.982261 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:54.982554 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.983050 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.983776 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:54.983952 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:54.984000 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:54.986504 (+ 2504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:43237: BlockingRecv error: recv got EOF from 127.3.15.131:43237 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 20:08:55 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:55 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:55.717496 12946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:55.712888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:56663)
0807 20:08:55.713052 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:55.713064 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:55.713122 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:55.713253 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:55.713254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:55.713425 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:55.713476 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:55.713478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.714136 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.714136 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:55.714671 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:55.714673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.715001 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.715002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:55.715046 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:55.715582 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:55.715587 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:55.717418 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":139,"threads_started":1}
W20260807 20:08:55.717613 12947 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:55.712963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:56663 (local address 127.3.15.190:39275)
0807 20:08:55.713127 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:55.713129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:55.713138 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:08:55.713163 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:55.713163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:55.713174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:55.713234 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:55.713561 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.714062 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.714749 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.714933 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.714984 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:55.717511 (+ 2527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:56663: BlockingRecv error: recv got EOF from 127.3.15.129:56663 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
Aug 07 20:08:55 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:55 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:55.851871 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:55.846867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:45669)
0807 20:08:55.847077 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:55.847091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:55.847169 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:55.847339 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:55.847341 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:55.847559 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:55.847608 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:55.847610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.848379 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.848380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:55.849013 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:55.849015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.849449 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.849450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:55.849517 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:55.849974 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:55.849979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:55.851807 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":109,"threads_started":1}
W20260807 20:08:55.852010 12947 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:55.846978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:45669 (local address 127.3.15.190:39275)
0807 20:08:55.847059 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:55.847062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:55.847145 (+ 83us) server_negotiation.cc:407] Connection header received
0807 20:08:55.847235 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:55.847236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:55.847250 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:55.847333 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:55.847701 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.848267 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.849108 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.849326 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.849448 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:55.851922 (+ 2474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:45669: BlockingRecv error: recv got EOF from 127.3.15.130:45669 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 20:08:55 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:55 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.131@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:55.991905 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:55.986984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.131:36237)
0807 20:08:55.987206 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:55.987215 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:55.987294 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:55.987447 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:55.987448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:55.987647 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:55.987697 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:55.987698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.988400 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.988402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:55.989013 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:55.989015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.989378 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.989379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:55.989445 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:55.989906 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:55.989910 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:55.991782 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":123,"threads_started":1}
W20260807 20:08:55.991971 12947 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:55.987051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:36237 (local address 127.3.15.190:39275)
0807 20:08:55.987147 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:55.987150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:55.987278 (+ 128us) server_negotiation.cc:407] Connection header received
0807 20:08:55.987367 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:55.987368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:55.987383 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:55.987432 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:55.987781 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.988297 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.989108 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:55.989303 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:55.989380 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:55.991881 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.131:36237: BlockingRecv error: recv got EOF from 127.3.15.131:36237 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 20:08:56 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:56 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.129@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:56.723048 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:56.718042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.129:41503)
0807 20:08:56.718219 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:56.718231 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:56.718294 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:56.718448 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:56.718448 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:56.718627 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:56.718683 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:56.718684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.719414 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.719415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:56.719975 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:56.719978 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.720367 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.720368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:56.720439 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:56.720954 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:56.720959 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:56.722930 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":92,"threads_started":1}
W20260807 20:08:56.723114 12951 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:56.718161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:41503 (local address 127.3.15.190:39275)
0807 20:08:56.718296 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:56.718298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:56.718326 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:08:56.718352 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:56.718352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:56.718363 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:56.718412 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:56.718756 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.719318 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.720072 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.720287 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.720366 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:56.723033 (+ 2667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.129:41503: BlockingRecv error: recv got EOF from 127.3.15.129:41503 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
Aug 07 20:08:56 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 20:08:56 dist-test-slave-1x4g krb5kdc[11840](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.15.130@KRBTEST.COM for oryx/127.3.15.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 20:08:56.857234 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:56.852410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39275 (local address 127.3.15.130:38877)
0807 20:08:56.852565 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:56.852582 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:56.852655 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:56.852837 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:56.852838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:56.853005 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:56.853056 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:56.853058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.853732 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.853733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:56.854258 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:56.854259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.854639 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.854640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:56.854680 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:56.855162 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:56.855167 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:56.857174 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.15.190:39275: Server oryx/127.3.15.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
W20260807 20:08:56.857380 12951 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:56.852491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:38877 (local address 127.3.15.190:39275)
0807 20:08:56.852554 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:56.852556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:56.852639 (+ 83us) server_negotiation.cc:407] Connection header received
0807 20:08:56.852703 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:56.852703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:56.852714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:56.852769 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:56.853123 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.853663 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.854337 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:56.854566 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:56.854628 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:56.857284 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.15.130:38877: BlockingRecv error: recv got EOF from 127.3.15.130:38877 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:08:56.903370 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12456
I20260807 20:08:56.911208 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12615
I20260807 20:08:56.917423 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12751
I20260807 20:08:56.923192 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12382
2026-08-07T20:08:56Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17614 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T20:08:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:56Z Disabled control of system clock
I20260807 20:08:56.954208 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42235
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:42235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 20:08:57.040478 12957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.040706 12957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.040728 12957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.041958 12957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:57.042008 12957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.042024 12957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.042043 12957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:57.042057 12957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:57.043475 12957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:42235
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.043744 12957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.044078 12957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.049609 12964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.049575 12963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.049587 12966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.049644 12957 server_base.cc:1034] running on GCE node
I20260807 20:08:57.050028 12957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.050328 12957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.051483 12957 hybrid_clock.cc:648] HybridClock initialized: now 1786133337051469 us; error 30 us; skew 500 ppm
I20260807 20:08:57.057319 12957 webserver.cc:460] Webserver started at http://127.3.15.190:33677/ using document root <none> and password file <none>
I20260807 20:08:57.057600 12957 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.057653 12957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.057835 12957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.059020 12957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "24da970957c44bbbaddbbf349ae8d89c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "1f22bf775c2a9849014899537793cff6"
server_key_iv: "d1de23b3f960dcfcfae3ae98c34d74c6"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.059418 12957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "24da970957c44bbbaddbbf349ae8d89c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "1f22bf775c2a9849014899537793cff6"
server_key_iv: "d1de23b3f960dcfcfae3ae98c34d74c6"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.060880 12957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 20:08:57.061738 12972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.061941 12957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:57.062058 12957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "24da970957c44bbbaddbbf349ae8d89c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "1f22bf775c2a9849014899537793cff6"
server_key_iv: "d1de23b3f960dcfcfae3ae98c34d74c6"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.062137 12957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.072352 12957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.072703 12957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.072937 12957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.076867 12957 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:42235
I20260807 20:08:57.076908 13025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:42235 every 8 connection(s)
I20260807 20:08:57.077294 12957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:57.077997 13026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:57.079195 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 12957
I20260807 20:08:57.079301 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:57.079509 3134 external_mini_cluster.cc:1428] Setting key 3508955d7600b2632b62b3795db9e5dc
I20260807 20:08:57.081610 13026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c: Bootstrap starting.
I20260807 20:08:57.082283 13026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:57.082597 13026 log.cc:824] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:57.083348 13026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c: No bootstrap required, opened a new log
I20260807 20:08:57.085289 13026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24da970957c44bbbaddbbf349ae8d89c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42235 } }
I20260807 20:08:57.085424 13026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:57.085448 13026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24da970957c44bbbaddbbf349ae8d89c, State: Initialized, Role: FOLLOWER
I20260807 20:08:57.085579 13026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24da970957c44bbbaddbbf349ae8d89c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42235 } }
I20260807 20:08:57.085671 13026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:57.085697 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.080070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48448 (local address 127.3.15.190:42235)
0807 20:08:57.080265 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.080273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.080298 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:08:57.080729 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.080735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.080865 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.080997 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.081235 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.082129 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.083185 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.083679 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.083785 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.084523 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.084527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.084527 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.084705 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.084742 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.084752 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.084926 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.085055 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.085172 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260807 20:08:57.085716 13026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:57.085812 13026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:57.086356 13026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24da970957c44bbbaddbbf349ae8d89c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42235 } }
I20260807 20:08:57.086448 13026 leader_election.cc:302] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24da970957c44bbbaddbbf349ae8d89c; no voters:
I20260807 20:08:57.086616 13026 leader_election.cc:288] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:57.086683 13031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:57.086854 13031 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [term 1 LEADER]: Becoming Leader. State: Replica: 24da970957c44bbbaddbbf349ae8d89c, State: Running, Role: LEADER
I20260807 20:08:57.086906 13026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:57.087023 13031 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24da970957c44bbbaddbbf349ae8d89c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42235 } }
I20260807 20:08:57.087420 13033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24da970957c44bbbaddbbf349ae8d89c. Latest consensus state: current_term: 1 leader_uuid: "24da970957c44bbbaddbbf349ae8d89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24da970957c44bbbaddbbf349ae8d89c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42235 } } }
I20260807 20:08:57.087529 13033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:57.087420 13032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24da970957c44bbbaddbbf349ae8d89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24da970957c44bbbaddbbf349ae8d89c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42235 } } }
I20260807 20:08:57.087795 13032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:57.088146 13039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:57.088637 13039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:57.090301 13039 catalog_manager.cc:1347] Generated new cluster ID: 75a1b84e38f8435a8084c8aafde71b63
I20260807 20:08:57.090354 13039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:57.101511 13039 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:57.102125 13039 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:57.106212 13039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 24da970957c44bbbaddbbf349ae8d89c: Generated new TSK 0
I20260807 20:08:57.106424 13039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:57.117221 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:42235
--builtin_ntp_servers=127.3.15.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 20:08:57.202786 13051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.203018 13051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.203043 13051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.204257 13051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.204304 13051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.204332 13051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:57.205833 13051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.206108 13051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.206449 13051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.212617 13057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.212878 13051 server_base.cc:1034] running on GCE node
W20260807 20:08:57.212663 13058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.212616 13060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.213274 13051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.213563 13051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.214740 13051 hybrid_clock.cc:648] HybridClock initialized: now 1786133337214725 us; error 30 us; skew 500 ppm
I20260807 20:08:57.216095 13051 webserver.cc:460] Webserver started at http://127.3.15.129:43019/ using document root <none> and password file <none>
I20260807 20:08:57.216352 13051 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.216424 13051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.216595 13051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.217801 13051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "feb8e51f25d144549c9bbe9f2e4d216c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "6c23c4acb80661c4aab003978f55a77a"
server_key_iv: "b9326141b5f012263d33e94315ff0359"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.218195 13051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "feb8e51f25d144549c9bbe9f2e4d216c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "6c23c4acb80661c4aab003978f55a77a"
server_key_iv: "b9326141b5f012263d33e94315ff0359"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.219594 13051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:57.220412 13066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.220593 13051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:57.220705 13051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "feb8e51f25d144549c9bbe9f2e4d216c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "6c23c4acb80661c4aab003978f55a77a"
server_key_iv: "b9326141b5f012263d33e94315ff0359"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.220817 13051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.244882 13051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.245257 13051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.245430 13051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.245712 13051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:57.246078 13051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:57.246135 13051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.246201 13051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:57.246230 13051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.252389 13051 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:34629
I20260807 20:08:57.252444 13185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:34629 every 8 connection(s)
I20260807 20:08:57.252843 13051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:57.253599 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13051
I20260807 20:08:57.253702 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:57.253916 3134 external_mini_cluster.cc:1428] Setting key 4609ee86922c4bee809a29bda57f8d50
I20260807 20:08:57.254956 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:42235
--builtin_ntp_servers=127.3.15.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 20:08:57.259179 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.253712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:53923 (local address 127.3.15.190:42235)
0807 20:08:57.253770 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.253773 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.254056 (+ 283us) server_negotiation.cc:407] Connection header received
0807 20:08:57.254606 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.254607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.254621 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.254670 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.255491 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.256007 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.257560 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.257791 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.257877 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.258892 (+ 1015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.258893 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.258894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.258898 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.258914 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.258918 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.258976 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.259086 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.259116 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:08:57.259518 13189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.253679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:42235 (local address 127.3.15.129:53923)
0807 20:08:57.253880 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:57.253924 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:57.254494 (+ 570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:57.254748 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:57.254753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:57.254767 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:57.255380 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:57.255386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.256120 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.256121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:57.257437 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:57.257441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.257896 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.257897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:57.258064 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.258702 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:57.258703 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:57.258747 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:57.258780 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:57.258995 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:57.259002 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:57.259005 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:57.259078 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:08:57.259215 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260807 20:08:57.260119 13186 heartbeater.cc:347] Connected to a master server at 127.3.15.190:42235
I20260807 20:08:57.260254 13186 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:57.260480 13186 heartbeater.cc:511] Master 127.3.15.190:42235 requested a full tablet report, sending...
I20260807 20:08:57.261161 12989 ts_manager.cc:194] Registered new tserver with Master: feb8e51f25d144549c9bbe9f2e4d216c (127.3.15.129:34629)
I20260807 20:08:57.262172 12989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.129:53923
W20260807 20:08:57.341173 13190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.341388 13190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.341413 13190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.342510 13190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.342543 13190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.342567 13190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:57.343827 13190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.344028 13190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.344283 13190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.349964 13195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.349982 13198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.350054 13196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.350363 13190 server_base.cc:1034] running on GCE node
I20260807 20:08:57.350591 13190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.350849 13190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.352031 13190 hybrid_clock.cc:648] HybridClock initialized: now 1786133337352018 us; error 30 us; skew 500 ppm
I20260807 20:08:57.353400 13190 webserver.cc:460] Webserver started at http://127.3.15.130:33513/ using document root <none> and password file <none>
I20260807 20:08:57.353672 13190 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.353721 13190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.353853 13190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.354933 13190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "35ab6230f34f4f98a950b98ae215e7d3"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "2a6d701b79fa2f720550fe932b7c9199"
server_key_iv: "87b1632f343513bd4d7b07a0149deda8"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.355302 13190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "35ab6230f34f4f98a950b98ae215e7d3"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "2a6d701b79fa2f720550fe932b7c9199"
server_key_iv: "87b1632f343513bd4d7b07a0149deda8"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.356691 13190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:08:57.357573 13205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.357779 13190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:57.357856 13190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "35ab6230f34f4f98a950b98ae215e7d3"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "2a6d701b79fa2f720550fe932b7c9199"
server_key_iv: "87b1632f343513bd4d7b07a0149deda8"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.357947 13190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.373323 13190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.373651 13190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.373808 13190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.374044 13190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:57.374419 13190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:57.374477 13190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.374519 13190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:57.374547 13190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.380668 13190 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:35925
I20260807 20:08:57.380709 13323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:35925 every 8 connection(s)
I20260807 20:08:57.381145 13190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:57.381677 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13190
I20260807 20:08:57.381805 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:57.382081 3134 external_mini_cluster.cc:1428] Setting key 00475a3153d005582f7ad4b90156bbb3
I20260807 20:08:57.383097 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:42235
--builtin_ntp_servers=127.3.15.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 20:08:57.387138 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.381993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46267 (local address 127.3.15.190:42235)
0807 20:08:57.382071 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.382074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.382374 (+ 300us) server_negotiation.cc:407] Connection header received
0807 20:08:57.383007 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.383008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.383020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.383062 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.383865 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.384376 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.385526 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.385747 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.385837 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.386766 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.386768 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.386768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.386774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.386791 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.386796 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.386869 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.387000 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.387033 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 20:08:57.387343 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.382001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:42235 (local address 127.3.15.130:46267)
0807 20:08:57.382227 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:57.382270 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:57.382897 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:57.383153 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:57.383158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:57.383175 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:57.383765 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:57.383771 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.384477 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.384478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:57.385428 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:57.385430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.385850 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.385851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:57.385997 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.386618 (+ 621us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:57.386618 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:57.386649 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:57.386675 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:57.386869 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:57.386876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:57.386880 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:08:57.386980 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 20:08:57.387095 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":118,"threads_started":1}
I20260807 20:08:57.387830 13324 heartbeater.cc:347] Connected to a master server at 127.3.15.190:42235
I20260807 20:08:57.387957 13324 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:57.388195 13324 heartbeater.cc:511] Master 127.3.15.190:42235 requested a full tablet report, sending...
I20260807 20:08:57.388607 12989 ts_manager.cc:194] Registered new tserver with Master: 35ab6230f34f4f98a950b98ae215e7d3 (127.3.15.130:35925)
I20260807 20:08:57.389053 12989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.130:46267
W20260807 20:08:57.469172 13328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.469416 13328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.469452 13328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.470620 13328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.470695 13328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.470741 13328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:57.472108 13328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:42235
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.472386 13328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.472697 13328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.477845 13334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.477854 13337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.477854 13335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.478205 13328 server_base.cc:1034] running on GCE node
I20260807 20:08:57.478443 13328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.478704 13328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.479873 13328 hybrid_clock.cc:648] HybridClock initialized: now 1786133337479847 us; error 42 us; skew 500 ppm
I20260807 20:08:57.481277 13328 webserver.cc:460] Webserver started at http://127.3.15.131:36425/ using document root <none> and password file <none>
I20260807 20:08:57.481526 13328 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.481597 13328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.481737 13328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.482820 13328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "0c62d76e02fb4f4380d00720f25e3e7c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "b84f3d93fc45496b13c075bc2ac01a22"
server_key_iv: "1af802f9e061216df16b9d4d90d6ddac"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.483206 13328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "0c62d76e02fb4f4380d00720f25e3e7c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "b84f3d93fc45496b13c075bc2ac01a22"
server_key_iv: "1af802f9e061216df16b9d4d90d6ddac"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.484642 13328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:57.485548 13343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.485740 13328 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:57.485848 13328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "0c62d76e02fb4f4380d00720f25e3e7c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "b84f3d93fc45496b13c075bc2ac01a22"
server_key_iv: "1af802f9e061216df16b9d4d90d6ddac"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.485939 13328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.504930 13328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.505263 13328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.505416 13328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.505666 13328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:57.506013 13328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:57.506065 13328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.506108 13328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:57.506150 13328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.512174 13328 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:40513
I20260807 20:08:57.512228 13456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:40513 every 8 connection(s)
I20260807 20:08:57.512629 13328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:57.518201 13029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.513444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:48145 (local address 127.3.15.190:42235)
0807 20:08:57.513492 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.513495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.513771 (+ 276us) server_negotiation.cc:407] Connection header received
0807 20:08:57.514322 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.514323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.514335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.514375 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.515087 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.515604 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.516649 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.516872 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.516976 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.517872 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.517873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.517874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.517879 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.517894 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.517898 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.517983 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.518092 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.518119 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:08:57.518452 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.513436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:42235 (local address 127.3.15.131:48145)
0807 20:08:57.513636 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:57.513672 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:57.514211 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:57.514466 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:57.514470 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:57.514484 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:57.514991 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:57.514997 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.515704 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.515705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:57.516534 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:57.516537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:57.516999 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:57.517000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:57.517134 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:57.517705 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:57.517706 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:57.517736 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:57.517762 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:57.517987 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:57.517994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:57.517997 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:57.518080 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:08:57.518190 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":84,"threads_started":1}
I20260807 20:08:57.518961 13458 heartbeater.cc:347] Connected to a master server at 127.3.15.190:42235
I20260807 20:08:57.519099 13458 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:57.519347 13458 heartbeater.cc:511] Master 127.3.15.190:42235 requested a full tablet report, sending...
I20260807 20:08:57.519378 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13328
I20260807 20:08:57.519475 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:57.519722 3134 external_mini_cluster.cc:1428] Setting key 926517b9d66f634139ea5f9600ea3008
I20260807 20:08:57.519912 12989 ts_manager.cc:194] Registered new tserver with Master: 0c62d76e02fb4f4380d00720f25e3e7c (127.3.15.131:40513)
I20260807 20:08:57.520354 12989 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.131:48145
I20260807 20:08:57.520581 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 20:08:57.522429 13029 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:57.522007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.3.15.190:42235)
0807 20:08:57.522090 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.522092 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.522163 (+ 71us) server_negotiation.cc:407] Connection header received
0807 20:08:57.522215 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.522216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.522224 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.522283 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.522384 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48464: BlockingRecv error: recv got EOF from 127.0.0.1:48464 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 20:08:57.522505 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13051
I20260807 20:08:57.529388 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13190
I20260807 20:08:57.535989 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13328
I20260807 20:08:57.544598 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 12957
2026-08-07T20:08:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (620 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T20:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:57Z Disabled control of system clock
I20260807 20:08:57.575294 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:41533
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:41533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:08:57.660157 13470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.660356 13470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.660377 13470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.661554 13470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:57.661595 13470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.661608 13470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.661646 13470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:57.661677 13470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:57.663043 13470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:41533
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:41533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.663303 13470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.663596 13470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.668475 13476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.668493 13478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.668494 13475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.668752 13470 server_base.cc:1034] running on GCE node
I20260807 20:08:57.669040 13470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.669339 13470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.670501 13470 hybrid_clock.cc:648] HybridClock initialized: now 1786133337670494 us; error 35 us; skew 500 ppm
I20260807 20:08:57.671823 13470 webserver.cc:460] Webserver started at http://127.3.15.190:34035/ using document root <none> and password file <none>
I20260807 20:08:57.672096 13470 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.672173 13470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.672333 13470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.673485 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "1ec2f4b43ef74155aba9cdc1c40d1465"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "4dddd09498fade31a318d7eff193be1b"
server_key_iv: "65839e9e1ba19cee54827de54517f1c3"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.673867 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "1ec2f4b43ef74155aba9cdc1c40d1465"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "4dddd09498fade31a318d7eff193be1b"
server_key_iv: "65839e9e1ba19cee54827de54517f1c3"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.675213 13470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:57.675940 13484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.676134 13470 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:57.676245 13470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "1ec2f4b43ef74155aba9cdc1c40d1465"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "4dddd09498fade31a318d7eff193be1b"
server_key_iv: "65839e9e1ba19cee54827de54517f1c3"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.676321 13470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.679819 13470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.680105 13470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.680248 13470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.684248 13470 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:41533
I20260807 20:08:57.684288 13537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:41533 every 8 connection(s)
I20260807 20:08:57.684737 13470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:57.685520 13538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:57.688591 13538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465: Bootstrap starting.
I20260807 20:08:57.689265 13538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:57.689580 13538 log.cc:824] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:57.690281 13538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465: No bootstrap required, opened a new log
I20260807 20:08:57.690304 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13470
I20260807 20:08:57.690447 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:57.690649 3134 external_mini_cluster.cc:1428] Setting key 67f7fabeb2d0f41b8932fdc5dbb99431
I20260807 20:08:57.692184 13538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41533 } }
I20260807 20:08:57.692322 13538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:57.692375 13538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec2f4b43ef74155aba9cdc1c40d1465, State: Initialized, Role: FOLLOWER
I20260807 20:08:57.692502 13538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41533 } }
I20260807 20:08:57.692581 13538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:57.692642 13538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:57.692700 13538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:57.693363 13538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41533 } }
I20260807 20:08:57.693509 13538 leader_election.cc:302] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ec2f4b43ef74155aba9cdc1c40d1465; no voters:
I20260807 20:08:57.693684 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.691185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.3.15.190:41533)
0807 20:08:57.691405 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.691414 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.691446 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:08:57.692164 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.692170 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.692218 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.692445 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.693137 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.693140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.693141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.693145 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.693192 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.693203 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.693283 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.693352 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.693411 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
I20260807 20:08:57.693711 13538 leader_election.cc:288] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:57.694010 13543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:57.694087 13538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:57.694154 13543 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [term 1 LEADER]: Becoming Leader. State: Replica: 1ec2f4b43ef74155aba9cdc1c40d1465, State: Running, Role: LEADER
I20260807 20:08:57.694242 13543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41533 } }
I20260807 20:08:57.694701 13545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ec2f4b43ef74155aba9cdc1c40d1465. Latest consensus state: current_term: 1 leader_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41533 } } }
I20260807 20:08:57.694707 13544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2f4b43ef74155aba9cdc1c40d1465" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 41533 } } }
I20260807 20:08:57.694932 13544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:57.694896 13545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:57.695295 13553 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:57.695895 13553 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:57.697410 13553 catalog_manager.cc:1347] Generated new cluster ID: 25d33012c666458fb2db500034795495
I20260807 20:08:57.697477 13553 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:08:57.723114 13553 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:57.723332 13553 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:57.727185 13553 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ec2f4b43ef74155aba9cdc1c40d1465: Generated new TSK 0
I20260807 20:08:57.727389 13553 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:08:57.737026 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:41533
--builtin_ntp_servers=127.3.15.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:08:57.825347 13563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.825567 13563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.825593 13563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.826727 13563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.826769 13563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.826829 13563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:57.828293 13563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.828522 13563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.828871 13563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.834021 13568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.834146 13569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.834070 13571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.834242 13563 server_base.cc:1034] running on GCE node
I20260807 20:08:57.834533 13563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.834808 13563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.835978 13563 hybrid_clock.cc:648] HybridClock initialized: now 1786133337835959 us; error 37 us; skew 500 ppm
I20260807 20:08:57.837572 13563 webserver.cc:460] Webserver started at http://127.3.15.129:39329/ using document root <none> and password file <none>
I20260807 20:08:57.837852 13563 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.837908 13563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.838096 13563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.839282 13563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "d5e344ecc2d34eb3a9efae84496d9b8c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "b0b2b1916709d66f6a74be1795e45a00"
server_key_iv: "b97817f11ebcc4ff535317554179af2d"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.839689 13563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "d5e344ecc2d34eb3a9efae84496d9b8c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "b0b2b1916709d66f6a74be1795e45a00"
server_key_iv: "b97817f11ebcc4ff535317554179af2d"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.841302 13563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 20:08:57.842103 13577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.842288 13563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:57.842389 13563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "d5e344ecc2d34eb3a9efae84496d9b8c"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "b0b2b1916709d66f6a74be1795e45a00"
server_key_iv: "b97817f11ebcc4ff535317554179af2d"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.842481 13563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.846283 13563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.846582 13563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.846733 13563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.846952 13563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:57.847282 13563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:57.847318 13563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.847378 13563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:57.847406 13563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.854063 13563 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:38679
I20260807 20:08:57.854115 13691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:38679 every 8 connection(s)
I20260807 20:08:57.854487 13563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:08:57.857576 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.855219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:37185 (local address 127.3.15.190:41533)
0807 20:08:57.855341 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.855344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.855559 (+ 215us) server_negotiation.cc:407] Connection header received
0807 20:08:57.856033 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.856034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.856043 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.856108 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.857328 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.857330 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.857330 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.857335 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.857353 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.857358 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.857417 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.857502 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.857518 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 20:08:57.857784 13694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.855214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:41533 (local address 127.3.15.129:37185)
0807 20:08:57.855422 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:57.855463 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:57.855942 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:57.856164 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:57.856167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:57.856175 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:57.857164 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:57.857164 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:57.857195 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:57.857225 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:57.857425 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:57.857430 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:57.857433 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:57.857493 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:08:57.857543 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260807 20:08:57.858310 13692 heartbeater.cc:347] Connected to a master server at 127.3.15.190:41533
I20260807 20:08:57.858417 13692 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:57.858579 13692 heartbeater.cc:511] Master 127.3.15.190:41533 requested a full tablet report, sending...
I20260807 20:08:57.859221 13501 ts_manager.cc:194] Registered new tserver with Master: d5e344ecc2d34eb3a9efae84496d9b8c (127.3.15.129:38679)
I20260807 20:08:57.862912 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13563
I20260807 20:08:57.862987 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:57.863198 3134 external_mini_cluster.cc:1428] Setting key 9a989bbb4d23fc45405e943dbfce702a
I20260807 20:08:57.864159 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:41533
--builtin_ntp_servers=127.3.15.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:08:57.948690 13695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:57.948969 13695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:57.948992 13695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:57.950258 13695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:57.950300 13695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:57.950326 13695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:57.951689 13695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:57.951938 13695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:57.952250 13695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:57.957412 13703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.957422 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:57.957422 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:57.957644 13695 server_base.cc:1034] running on GCE node
I20260807 20:08:57.957840 13695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:57.958101 13695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:57.959272 13695 hybrid_clock.cc:648] HybridClock initialized: now 1786133337959258 us; error 50 us; skew 500 ppm
I20260807 20:08:57.960631 13695 webserver.cc:460] Webserver started at http://127.3.15.130:36997/ using document root <none> and password file <none>
I20260807 20:08:57.960944 13695 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:57.961017 13695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:57.961167 13695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:57.962368 13695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "eb8de29b40494d8cbe6ea901a6c9d857"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "c03f7be1ef98f047ac1a17f6502c0b3e"
server_key_iv: "2f9e28e46119fb716395cb75b21d6c63"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.962785 13695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "eb8de29b40494d8cbe6ea901a6c9d857"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "c03f7be1ef98f047ac1a17f6502c0b3e"
server_key_iv: "2f9e28e46119fb716395cb75b21d6c63"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.964296 13695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:57.965212 13709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.965417 13695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:08:57.965528 13695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "eb8de29b40494d8cbe6ea901a6c9d857"
format_stamp: "Formatted at 2026-08-07 20:08:57 on dist-test-slave-1x4g"
server_key: "c03f7be1ef98f047ac1a17f6502c0b3e"
server_key_iv: "2f9e28e46119fb716395cb75b21d6c63"
server_key_version: "encryptionkey@0"
I20260807 20:08:57.965623 13695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:57.969121 13695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:57.969406 13695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:57.969574 13695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:57.969813 13695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:57.970115 13695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:57.970149 13695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.970171 13695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:57.970183 13695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:57.976131 13695 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:44301
I20260807 20:08:57.976284 13823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:44301 every 8 connection(s)
I20260807 20:08:57.976591 13695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:57.979893 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13695
I20260807 20:08:57.980006 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:57.980099 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.977492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:50929 (local address 127.3.15.190:41533)
0807 20:08:57.977562 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:57.977564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:57.977811 (+ 247us) server_negotiation.cc:407] Connection header received
0807 20:08:57.978376 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:57.978377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:57.978385 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:57.978437 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:57.979812 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:57.979814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:57.979814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:57.979819 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:57.979839 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:57.979843 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:57.979902 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:57.980006 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 20:08:57.980025 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:08:57.980258 3134 external_mini_cluster.cc:1428] Setting key ea1551cbc5b2da6d86303ddc7a062114
I20260807 20:08:57.980347 13826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:57.977472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:41533 (local address 127.3.15.130:50929)
0807 20:08:57.977682 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:57.977717 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:57.978243 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:57.978555 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:57.978559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:57.978567 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:57.979648 (+ 1081us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:57.979649 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:57.979682 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:57.979714 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:57.979911 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:57.979917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:57.979920 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:08:57.980016 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 20:08:57.980084 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":84,"threads_started":1}
I20260807 20:08:57.980873 13824 heartbeater.cc:347] Connected to a master server at 127.3.15.190:41533
I20260807 20:08:57.980971 13824 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:57.981149 13824 heartbeater.cc:511] Master 127.3.15.190:41533 requested a full tablet report, sending...
I20260807 20:08:57.981565 13500 ts_manager.cc:194] Registered new tserver with Master: eb8de29b40494d8cbe6ea901a6c9d857 (127.3.15.130:44301)
I20260807 20:08:57.981664 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:41533
--builtin_ntp_servers=127.3.15.148:43519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 20:08:58.065536 13827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:58.065742 13827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:58.065764 13827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:58.066880 13827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:58.066915 13827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:58.066939 13827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:58.068349 13827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:43519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:41533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:58.068535 13827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:58.068763 13827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:58.073904 13833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:58.073966 13832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:58.073904 13835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:58.074103 13827 server_base.cc:1034] running on GCE node
I20260807 20:08:58.074333 13827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:58.074606 13827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:58.075759 13827 hybrid_clock.cc:648] HybridClock initialized: now 1786133338075746 us; error 31 us; skew 500 ppm
I20260807 20:08:58.077081 13827 webserver.cc:460] Webserver started at http://127.3.15.131:40957/ using document root <none> and password file <none>
I20260807 20:08:58.077334 13827 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:58.077380 13827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:58.077545 13827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:58.078704 13827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "22afe2e9a3b64e4690969427dfab2925"
format_stamp: "Formatted at 2026-08-07 20:08:58 on dist-test-slave-1x4g"
server_key: "b770b3efa48aedae4638a43fe7f8f365"
server_key_iv: "7ed4745450b71cbf90fc65099f83e9d5"
server_key_version: "encryptionkey@0"
I20260807 20:08:58.079033 13827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "22afe2e9a3b64e4690969427dfab2925"
format_stamp: "Formatted at 2026-08-07 20:08:58 on dist-test-slave-1x4g"
server_key: "b770b3efa48aedae4638a43fe7f8f365"
server_key_iv: "7ed4745450b71cbf90fc65099f83e9d5"
server_key_version: "encryptionkey@0"
I20260807 20:08:58.080428 13827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:58.081261 13841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:58.081434 13827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:08:58.081544 13827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "22afe2e9a3b64e4690969427dfab2925"
format_stamp: "Formatted at 2026-08-07 20:08:58 on dist-test-slave-1x4g"
server_key: "b770b3efa48aedae4638a43fe7f8f365"
server_key_iv: "7ed4745450b71cbf90fc65099f83e9d5"
server_key_version: "encryptionkey@0"
I20260807 20:08:58.081620 13827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:58.084939 13827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:58.085206 13827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:58.085350 13827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:58.085563 13827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:58.085875 13827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:58.085930 13827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:58.085969 13827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:58.085997 13827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:58.091717 13827 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:37531
I20260807 20:08:58.091789 13958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:37531 every 8 connection(s)
I20260807 20:08:58.092110 13827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:08:58.095325 13542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:58.092847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:59215 (local address 127.3.15.190:41533)
0807 20:08:58.092901 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:58.092903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:58.093173 (+ 270us) server_negotiation.cc:407] Connection header received
0807 20:08:58.093702 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:58.093703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:58.093709 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:58.093760 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:58.095019 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:58.095020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:58.095020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:08:58.095023 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:58.095035 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:08:58.095039 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:58.095084 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:58.095212 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:08:58.095224 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:08:58.095418 13962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:58.092842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:41533 (local address 127.3.15.131:59215)
0807 20:08:58.093031 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:58.093091 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:58.093609 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:58.093812 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:58.093815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:58.093823 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:58.094882 (+ 1059us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:08:58.094883 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:58.094911 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:08:58.094945 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:58.095081 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:58.095084 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:58.095105 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 20:08:58.095209 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 20:08:58.095262 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260807 20:08:58.095903 13959 heartbeater.cc:347] Connected to a master server at 127.3.15.190:41533
I20260807 20:08:58.095997 13959 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:58.096105 13959 heartbeater.cc:511] Master 127.3.15.190:41533 requested a full tablet report, sending...
I20260807 20:08:58.096426 13500 ts_manager.cc:194] Registered new tserver with Master: 22afe2e9a3b64e4690969427dfab2925 (127.3.15.131:37531)
I20260807 20:08:58.097254 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13827
I20260807 20:08:58.097328 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:08:58.097491 3134 external_mini_cluster.cc:1428] Setting key 9d5a99c58ea0c7846c128e15cdd2d94f
I20260807 20:08:58.098193 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 20:08:58.099838 13542 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:08:58.099413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.3.15.190:41533)
0807 20:08:58.099480 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:58.099482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:58.099570 (+ 88us) server_negotiation.cc:407] Connection header received
0807 20:08:58.099624 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:58.099625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:58.099629 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:58.099675 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:58.099763 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58138: BlockingRecv error: recv got EOF from 127.0.0.1:58138 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 20:08:58.099943 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13563
I20260807 20:08:58.111220 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13695
I20260807 20:08:58.118213 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13827
I20260807 20:08:58.123121 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13470
2026-08-07T20:08:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (578 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:08:58 dist-test-slave-1x4g krb5kdc[13970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:08:58 dist-test-slave-1x4g krb5kdc[13970](info): set up 2 sockets
Aug 07 20:08:58 dist-test-slave-1x4g krb5kdc[13970](info): commencing operation
krb5kdc: starting...
W20260807 20:08:59.199077 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:08:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:08:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:08:59.328640 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:43919
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:43919
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:59.413295 13984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:59.413522 13984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:59.413545 13984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:59.414759 13984 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:08:59.414805 13984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:59.414819 13984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:59.414834 13984 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:08:59.414845 13984 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:08:59.416424 13984 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:43919
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:43919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:59.416725 13984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:59.417026 13984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:59.422135 13990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:59.422135 13992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:59.422214 13989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:59.422165 13984 server_base.cc:1034] running on GCE node
I20260807 20:08:59.422600 13984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:59.422925 13984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:59.424124 13984 hybrid_clock.cc:648] HybridClock initialized: now 1786133339424114 us; error 40 us; skew 500 ppm
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:59.426932 13984 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:08:59.427331 13984 webserver.cc:460] Webserver started at http://127.3.15.190:42337/ using document root <none> and password file <none>
I20260807 20:08:59.427534 13984 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:59.427577 13984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:59.427662 13984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:59.428886 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "4fcf6e7892fd4ca584a848c67ca1de91"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "e337ee6a03d1afa99d80f3eb016136e3"
server_key_iv: "787197aa90fc4049a2a3c81eced78625"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.429232 13984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "4fcf6e7892fd4ca584a848c67ca1de91"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "e337ee6a03d1afa99d80f3eb016136e3"
server_key_iv: "787197aa90fc4049a2a3c81eced78625"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.430524 13984 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:59.431355 13999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.431581 13984 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:08:59.431686 13984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "4fcf6e7892fd4ca584a848c67ca1de91"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "e337ee6a03d1afa99d80f3eb016136e3"
server_key_iv: "787197aa90fc4049a2a3c81eced78625"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.431759 13984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:59.459142 13984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:59.461515 13984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:59.461649 13984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:59.466148 13984 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:43919
I20260807 20:08:59.466207 14053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:43919 every 8 connection(s)
I20260807 20:08:59.466596 13984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:08:59.467676 14054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:08:59.470086 14054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91: Bootstrap starting.
I20260807 20:08:59.470714 14054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91: Neither blocks nor log segments found. Creating new log.
I20260807 20:08:59.471024 14054 log.cc:824] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91: Log is configured to *not* fsync() on all Append() calls
I20260807 20:08:59.471762 14054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91: No bootstrap required, opened a new log
I20260807 20:08:59.473524 14054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 43919 } }
I20260807 20:08:59.473691 14054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:08:59.473739 14054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fcf6e7892fd4ca584a848c67ca1de91, State: Initialized, Role: FOLLOWER
I20260807 20:08:59.473915 14054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 43919 } }
I20260807 20:08:59.474009 14054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:08:59.474049 14054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:08:59.474103 14054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:08:59.474740 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 13984
I20260807 20:08:59.474869 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:08:59.475085 3134 external_mini_cluster.cc:1428] Setting key c91dc44029fb8583b7aad9c12b4b1cc9
I20260807 20:08:59.475234 14054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 43919 } }
I20260807 20:08:59.475371 14054 leader_election.cc:302] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fcf6e7892fd4ca584a848c67ca1de91; no voters:
I20260807 20:08:59.475560 14054 leader_election.cc:288] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:08:59.475584 14057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:08:59.475706 14057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [term 1 LEADER]: Becoming Leader. State: Replica: 4fcf6e7892fd4ca584a848c67ca1de91, State: Running, Role: LEADER
I20260807 20:08:59.475839 14054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:08:59.475858 14057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 43919 } }
I20260807 20:08:59.476213 14057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fcf6e7892fd4ca584a848c67ca1de91. Latest consensus state: current_term: 1 leader_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 43919 } } }
I20260807 20:08:59.476281 14057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:59.476256 14061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fcf6e7892fd4ca584a848c67ca1de91" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 43919 } } }
I20260807 20:08:59.476348 14061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91 [sys.catalog]: This master's current role is: LEADER
I20260807 20:08:59.476497 14064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:08:59.477172 14064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:08:59.478848 14064 catalog_manager.cc:1347] Generated new cluster ID: df4188c3a4c64a1d8c489d692e9ea27c
I20260807 20:08:59.478902 14064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:59.488471 14064 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:08:59.488605 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.475824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54438 (local address 127.3.15.190:43919)
0807 20:08:59.475989 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:59.475999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:59.476131 (+ 132us) server_negotiation.cc:407] Connection header received
0807 20:08:59.476290 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:59.476298 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:59.476361 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:59.477406 (+ 1045us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:59.477524 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.478995 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.480083 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.480694 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.480824 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:59.483715 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:59.483720 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:59.483721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:59.483734 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:59.486639 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:59.487078 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:59.487082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:59.487083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:59.487214 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:59.487450 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:59.487451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:59.487451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:59.487761 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:59.487954 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:59.488174 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 20:08:59.488333 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260807 20:08:59.489216 14064 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:08:59.492457 14064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91: Generated new TSK 0
I20260807 20:08:59.492624 14064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:08:59.523303 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:43919
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:59.606777 14081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:59.607008 14081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:59.607046 14081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:59.608188 14081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:59.608255 14081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:59.608299 14081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:08:59.609781 14081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:43919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:59.610042 14081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:59.610278 14081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:59.615454 14089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:59.615484 14086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:59.615556 14087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:59.615630 14081 server_base.cc:1034] running on GCE node
I20260807 20:08:59.615957 14081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:59.616185 14081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:59.617331 14081 hybrid_clock.cc:648] HybridClock initialized: now 1786133339617312 us; error 34 us; skew 500 ppm
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:59.619760 14081 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:08:59.620184 14081 webserver.cc:460] Webserver started at http://127.3.15.129:46119/ using document root <none> and password file <none>
I20260807 20:08:59.620414 14081 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:59.620481 14081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:59.620638 14081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:59.621778 14081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "c79b8926577c480e8240de46fe9e2a53"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "4ebb4b135ef630c9348eea7f3e46cd41"
server_key_iv: "8357c2ef5e933fce59dda1157f7d2e2f"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.622170 14081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "c79b8926577c480e8240de46fe9e2a53"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "4ebb4b135ef630c9348eea7f3e46cd41"
server_key_iv: "8357c2ef5e933fce59dda1157f7d2e2f"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.623497 14081 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:59.624240 14098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.624423 14081 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:59.624529 14081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "c79b8926577c480e8240de46fe9e2a53"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "4ebb4b135ef630c9348eea7f3e46cd41"
server_key_iv: "8357c2ef5e933fce59dda1157f7d2e2f"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.624621 14081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:59.653532 14081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:59.655579 14081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:59.655761 14081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:59.656015 14081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:59.656358 14081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:59.656415 14081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.656461 14081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:59.656491 14081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.662570 14081 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:38131
I20260807 20:08:59.662621 14216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:38131 every 8 connection(s)
I20260807 20:08:59.662954 14081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:59.669534 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 14081
I20260807 20:08:59.669668 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:08:59.669929 3134 external_mini_cluster.cc:1428] Setting key 6491613974dc1ae31ea4c055146ce76b
I20260807 20:08:59.672847 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.663659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:57883 (local address 127.3.15.190:43919)
0807 20:08:59.663734 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:59.663736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:59.663958 (+ 222us) server_negotiation.cc:407] Connection header received
0807 20:08:59.664471 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:59.664471 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:59.664481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:59.664540 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:59.665552 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.666053 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.667027 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.667235 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.667289 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:59.669574 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:59.669575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:59.669575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:59.669582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:59.671191 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:59.671710 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:59.671711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:59.671711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:59.671765 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:59.672056 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:59.672057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:59.672057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:59.672166 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:59.672220 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:59.672573 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 20:08:59.672692 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:08:59.673065 14219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.663645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:43919 (local address 127.3.15.129:57883)
0807 20:08:59.663827 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:59.663870 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:59.664383 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:59.664594 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:59.664598 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:59.665258 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:59.665471 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:59.665476 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.666142 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.666143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:59.666953 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:59.666955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.667310 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.667311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:59.667414 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:59.667956 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:59.667966 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:59.669468 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:59.671277 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:59.671285 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:59.671287 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:59.671631 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:59.671831 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:59.671832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:59.671833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:59.671992 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:59.672234 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:59.672240 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:59.672417 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 20:08:59.672538 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 20:08:59.672739 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260807 20:08:59.673552 14217 heartbeater.cc:347] Connected to a master server at 127.3.15.190:43919
I20260807 20:08:59.673642 14217 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:59.673841 14217 heartbeater.cc:511] Master 127.3.15.190:43919 requested a full tablet report, sending...
I20260807 20:08:59.674417 14016 ts_manager.cc:194] Registered new tserver with Master: c79b8926577c480e8240de46fe9e2a53 (127.3.15.129:38131)
I20260807 20:08:59.675407 14016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:57883
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:08:59.699441 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:43919
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:59.783627 14222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:59.783849 14222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:59.783874 14222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:59.784986 14222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:59.785024 14222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:59.785050 14222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:08:59.786445 14222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:43919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:59.786633 14222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:59.786815 14222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:59.791872 14227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:59.791970 14230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:59.791914 14222 server_base.cc:1034] running on GCE node
W20260807 20:08:59.791955 14228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:59.792297 14222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:59.792548 14222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:59.793725 14222 hybrid_clock.cc:648] HybridClock initialized: now 1786133339793719 us; error 39 us; skew 500 ppm
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:59.796201 14222 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:08:59.796686 14222 webserver.cc:460] Webserver started at http://127.3.15.130:35537/ using document root <none> and password file <none>
I20260807 20:08:59.796936 14222 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:59.796985 14222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:59.797078 14222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:59.798166 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "57cca4fbd0d74d789ae832a899104b0d"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "06e8b9618c7842c7a20b9b1835e79fb1"
server_key_iv: "be2d0baeeddbe49a09ffcf797f566a4f"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.798475 14222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "57cca4fbd0d74d789ae832a899104b0d"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "06e8b9618c7842c7a20b9b1835e79fb1"
server_key_iv: "be2d0baeeddbe49a09ffcf797f566a4f"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.799774 14222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:08:59.800529 14239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.800727 14222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:08:59.800849 14222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "57cca4fbd0d74d789ae832a899104b0d"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "06e8b9618c7842c7a20b9b1835e79fb1"
server_key_iv: "be2d0baeeddbe49a09ffcf797f566a4f"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.800957 14222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:59.822611 14222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:59.825505 14222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:59.825678 14222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:59.825917 14222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:59.826252 14222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:59.826308 14222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.826356 14222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:59.826383 14222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.832479 14222 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:34175
I20260807 20:08:59.832552 14356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:34175 every 8 connection(s)
I20260807 20:08:59.832957 14222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:08:59.835785 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 14222
I20260807 20:08:59.835955 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:08:59.836182 3134 external_mini_cluster.cc:1428] Setting key 2cc2934ba65268ed8821b1321fcdb59b
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:08:59.843986 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.833761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56983 (local address 127.3.15.190:43919)
0807 20:08:59.833811 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:59.833813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:08:59.833989 (+ 176us) server_negotiation.cc:407] Connection header received
0807 20:08:59.834568 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:08:59.834569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:08:59.834579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:08:59.834643 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:08:59.835672 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.836445 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.837965 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.838157 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.838217 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:59.840724 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:08:59.840726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:08:59.840726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:08:59.840735 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:08:59.842505 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:59.842920 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:59.842922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:59.842922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:59.843014 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:08:59.843281 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:08:59.843282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:08:59.843283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:08:59.843438 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:08:59.843526 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 20:08:59.843777 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 20:08:59.843873 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:08:59.844024 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.833677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:43919 (local address 127.3.15.130:56983)
0807 20:08:59.833840 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:08:59.833883 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:08:59.834478 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:08:59.834703 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:08:59.834706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:08:59.835359 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:08:59.835576 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:59.835583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.836560 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.836562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:59.837864 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:08:59.837867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:08:59.838246 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:08:59.838247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:08:59.838397 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:08:59.839026 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:08:59.839035 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:08:59.840624 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:08:59.842604 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:59.842611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:59.842613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:59.842843 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:59.843087 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:08:59.843088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:08:59.843088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:08:59.843209 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:08:59.843518 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:08:59.843522 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:08:59.843645 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 20:08:59.843731 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 20:08:59.843858 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260807 20:08:59.844573 14357 heartbeater.cc:347] Connected to a master server at 127.3.15.190:43919
I20260807 20:08:59.844694 14357 heartbeater.cc:464] Registering TS with master...
I20260807 20:08:59.844902 14357 heartbeater.cc:511] Master 127.3.15.190:43919 requested a full tablet report, sending...
I20260807 20:08:59.845369 14016 ts_manager.cc:194] Registered new tserver with Master: 57cca4fbd0d74d789ae832a899104b0d (127.3.15.130:34175)
I20260807 20:08:59.845916 14016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:56983
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:08:59.865883 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:43919
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:08:59.950084 14362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:08:59.950282 14362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:08:59.950305 14362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:08:59.951503 14362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:08:59.951545 14362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:08:59.951570 14362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:08:59.953058 14362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:43919
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:08:59.953302 14362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:08:59.953612 14362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:08:59.958700 14367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:08:59.958711 14368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:59.958837 14362 server_base.cc:1034] running on GCE node
W20260807 20:08:59.958750 14370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:08:59.959105 14362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:08:59.959311 14362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:08:59.960513 14362 hybrid_clock.cc:648] HybridClock initialized: now 1786133339960500 us; error 29 us; skew 500 ppm
Aug 07 20:08:59 dist-test-slave-1x4g krb5kdc[13970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:08:59.963203 14362 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:08:59.963593 14362 webserver.cc:460] Webserver started at http://127.3.15.131:45003/ using document root <none> and password file <none>
I20260807 20:08:59.963814 14362 fs_manager.cc:359] Metadata directory not provided
I20260807 20:08:59.963857 14362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:08:59.963961 14362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:08:59.965067 14362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "6b774a0ae27b46d885f5c3a7a41ff27b"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "a1130ea1fde90cdd8fb49713b047af62"
server_key_iv: "6060346a9e796b000efd98339aa6c723"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.965384 14362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "6b774a0ae27b46d885f5c3a7a41ff27b"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "a1130ea1fde90cdd8fb49713b047af62"
server_key_iv: "6060346a9e796b000efd98339aa6c723"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.966634 14362 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:08:59.967368 14380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.967538 14362 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:08:59.967643 14362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "6b774a0ae27b46d885f5c3a7a41ff27b"
format_stamp: "Formatted at 2026-08-07 20:08:59 on dist-test-slave-1x4g"
server_key: "a1130ea1fde90cdd8fb49713b047af62"
server_key_iv: "6060346a9e796b000efd98339aa6c723"
server_key_version: "encryptionkey@0"
I20260807 20:08:59.967717 14362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:08:59.989935 14362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:08:59.991956 14362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:08:59.992138 14362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:08:59.992410 14362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:08:59.992878 14362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:08:59.992944 14362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.992980 14362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:08:59.993005 14362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:08:59.998744 14362 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:44699
I20260807 20:08:59.998791 14499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:44699 every 8 connection(s)
I20260807 20:08:59.999127 14362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:09:00.001690 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 14362
I20260807 20:09:00.001803 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:09:00.001989 3134 external_mini_cluster.cc:1428] Setting key 8b39248bd7c326f7a59ebd399a6d8548
Aug 07 20:09:00 dist-test-slave-1x4g krb5kdc[13970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133339, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:09:00.008904 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.999855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:48211 (local address 127.3.15.190:43919)
0807 20:08:59.999902 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:08:59.999904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.000218 (+ 314us) server_negotiation.cc:407] Connection header received
0807 20:09:00.000643 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.000645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.000656 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.000725 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:00.001751 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.002396 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.003604 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.003866 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.003922 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.006169 (+ 2247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:00.006170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:00.006171 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:00.006181 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:00.007449 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:00.007872 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:00.007873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:00.007873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:00.007937 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:00.008208 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:00.008209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:00.008209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:00.008324 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:00.008396 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.008691 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.008812 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:09:00.009074 14502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:08:59.999820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:43919 (local address 127.3.15.131:48211)
0807 20:08:59.999986 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.000030 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:00.000546 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:00.000833 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:00.000838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:00.001443 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:09:00.001647 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.001652 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.002476 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.002477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.003526 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.003529 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.003974 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.003975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.004082 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.004521 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:09:00.004529 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:09:00.006061 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:09:00.007547 (+ 1486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:00.007554 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:00.007556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:00.007792 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:00.008013 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:00.008014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:00.008015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:00.008131 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:00.008399 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:09:00.008406 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:09:00.008546 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 20:09:00.008653 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 20:09:00.008836 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
I20260807 20:09:00.009544 14500 heartbeater.cc:347] Connected to a master server at 127.3.15.190:43919
I20260807 20:09:00.009645 14500 heartbeater.cc:464] Registering TS with master...
I20260807 20:09:00.009795 14500 heartbeater.cc:511] Master 127.3.15.190:43919 requested a full tablet report, sending...
I20260807 20:09:00.010131 14016 ts_manager.cc:194] Registered new tserver with Master: 6b774a0ae27b46d885f5c3a7a41ff27b (127.3.15.131:44699)
I20260807 20:09:00.010521 14016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:48211
I20260807 20:09:00.013638 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:09:00.021262 14062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.014956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.3.15.190:43919)
0807 20:09:00.015036 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.015038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.015170 (+ 132us) server_negotiation.cc:407] Connection header received
0807 20:09:00.015209 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.015210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.015218 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.015264 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:00.015817 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.016328 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.017120 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.017304 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.017362 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.018436 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:00.018437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:00.018438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:00.018445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:00.019991 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:00.020303 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:00.020304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:00.020304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:00.020364 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:00.020549 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:00.020550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:00.020550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:00.020715 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:00.021026 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.021093 (+ 67us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.021186 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 20:09:00.022894 14016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:09:00.024583 14016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:09:00.031772 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.027327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34608 (local address 127.3.15.129:38131)
0807 20:09:00.027546 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.027550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.028186 (+ 636us) server_negotiation.cc:407] Connection header received
0807 20:09:00.028230 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.028231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.028244 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.028327 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:00.028429 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.029464 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.030737 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.031281 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.031375 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.031459 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.031635 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.031680 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":104,"threads_started":1}
I20260807 20:09:00.031791 14513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.027062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:38131 (local address 127.0.0.1:34608)
0807 20:09:00.028071 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.028081 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:00.028129 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:00.028325 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:00.028326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:00.028328 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:00.028365 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.028366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.029583 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.029584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.030672 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.030674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.031397 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.031398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.031490 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.031506 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 20:09:00.031610 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 20:09:00.031676 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"thread_start_us":104,"threads_started":1}
I20260807 20:09:00.031814 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.027250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50466 (local address 127.3.15.131:44699)
0807 20:09:00.027462 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.027467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.027480 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:09:00.027568 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.027571 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.027587 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.027670 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:00.027897 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.029109 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.030641 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.031325 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.031388 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.031446 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.031686 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.031729 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260807 20:09:00.031802 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.027058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:34175 (local address 127.0.0.1:33466)
0807 20:09:00.027607 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.027617 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:00.027691 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:00.027972 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:00.027973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:00.027976 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:00.028017 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.028018 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.029467 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.029468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.030395 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.030398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.031488 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.031488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.031579 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.031585 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:09:00.031642 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 20:09:00.031699 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":6,"thread_start_us":85,"threads_started":1}
I20260807 20:09:00.031989 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.027451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.3.15.130:34175)
0807 20:09:00.027811 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.027816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.027824 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:09:00.027862 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.027865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.027882 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.027972 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:00.028124 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.029361 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.030488 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.031311 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.031388 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.031446 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.031839 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.031887 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":77,"threads_started":1}
I20260807 20:09:00.032186 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.027058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:44699 (local address 127.0.0.1:50466)
0807 20:09:00.027340 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.027384 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:00.027471 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:00.027711 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:00.027715 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:00.027717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:00.027831 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.027834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.029205 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.029207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.030542 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.030545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.031486 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.031487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.031579 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.031584 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:09:00.031638 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 20:09:00.032104 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":476,"thread_start_us":74,"threads_started":1}
I20260807 20:09:00.032809 14290 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8a919582c4e528c5bc891f0c4b331 (DEFAULT_TABLE table=test-table [id=b266e12acf0c4d978adf7ccfd12d2202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:09:00.032974 14431 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8a919582c4e528c5bc891f0c4b331 (DEFAULT_TABLE table=test-table [id=b266e12acf0c4d978adf7ccfd12d2202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:09:00.033171 14290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8a919582c4e528c5bc891f0c4b331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:00.033317 14431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8a919582c4e528c5bc891f0c4b331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:00.033257 14148 tablet_service.cc:1467] Processing CreateTablet for tablet 06c8a919582c4e528c5bc891f0c4b331 (DEFAULT_TABLE table=test-table [id=b266e12acf0c4d978adf7ccfd12d2202]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:09:00.033566 14148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c8a919582c4e528c5bc891f0c4b331. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:00.036127 14518 tablet_bootstrap.cc:492] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Bootstrap starting.
I20260807 20:09:00.036688 14519 tablet_bootstrap.cc:492] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: Bootstrap starting.
I20260807 20:09:00.036844 14518 tablet_bootstrap.cc:654] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:00.037424 14519 tablet_bootstrap.cc:654] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:00.037446 14518 log.cc:824] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:00.037997 14519 log.cc:824] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:00.038291 14518 tablet_bootstrap.cc:492] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: No bootstrap required, opened a new log
I20260807 20:09:00.038343 14520 tablet_bootstrap.cc:492] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Bootstrap starting.
I20260807 20:09:00.038367 14518 ts_tablet_manager.cc:1397] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:09:00.039014 14520 tablet_bootstrap.cc:654] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:00.039554 14520 log.cc:824] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:00.040369 14519 tablet_bootstrap.cc:492] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: No bootstrap required, opened a new log
I20260807 20:09:00.040465 14519 ts_tablet_manager.cc:1397] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 20:09:00.040428 14518 raft_consensus.cc:348] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.040624 14518 raft_consensus.cc:374] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:00.040652 14518 raft_consensus.cc:729] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57cca4fbd0d74d789ae832a899104b0d, State: Initialized, Role: FOLLOWER
I20260807 20:09:00.040817 14518 consensus_queue.cc:260] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.041137 14518 ts_tablet_manager.cc:1428] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 20:09:00.041189 14520 tablet_bootstrap.cc:492] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: No bootstrap required, opened a new log
I20260807 20:09:00.041247 14520 ts_tablet_manager.cc:1397] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:09:00.041268 14357 heartbeater.cc:503] Master 127.3.15.190:43919 was elected leader, sending a full tablet report...
I20260807 20:09:00.042984 14520 raft_consensus.cc:348] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.042925 14519 raft_consensus.cc:348] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.043107 14520 raft_consensus.cc:374] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:00.043128 14520 raft_consensus.cc:729] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b774a0ae27b46d885f5c3a7a41ff27b, State: Initialized, Role: FOLLOWER
I20260807 20:09:00.043133 14519 raft_consensus.cc:374] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:00.043171 14519 raft_consensus.cc:729] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79b8926577c480e8240de46fe9e2a53, State: Initialized, Role: FOLLOWER
I20260807 20:09:00.043239 14520 consensus_queue.cc:260] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.043295 14519 consensus_queue.cc:260] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.043509 14520 ts_tablet_manager.cc:1428] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:09:00.043627 14519 ts_tablet_manager.cc:1428] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:09:00.043650 14217 heartbeater.cc:503] Master 127.3.15.190:43919 was elected leader, sending a full tablet report...
I20260807 20:09:00.043651 14500 heartbeater.cc:503] Master 127.3.15.190:43919 was elected leader, sending a full tablet report...
W20260807 20:09:00.083580 14358 tablet.cc:2369] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:09:00.100292 14524 raft_consensus.cc:482] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:09:00.100446 14524 raft_consensus.cc:504] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.100867 14524 leader_election.cc:288] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b774a0ae27b46d885f5c3a7a41ff27b (127.3.15.131:44699), c79b8926577c480e8240de46fe9e2a53 (127.3.15.129:38131)
I20260807 20:09:00.103892 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.101137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:38125 (local address 127.3.15.131:44699)
0807 20:09:00.101207 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.101209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.101221 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:09:00.101300 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.101301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.101311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.101377 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:00.101558 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.102044 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.103020 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.103538 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.103621 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.103631 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.103791 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.103828 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:09:00.103892 14359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.101061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:44699 (local address 127.3.15.130:38125)
0807 20:09:00.101152 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.101166 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:00.101219 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:00.101400 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:00.101401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:00.101404 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:00.101485 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.101486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.102140 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.102142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.102925 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.102928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.103619 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.103620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.103708 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.103711 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:09:00.103773 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 20:09:00.103817 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":30}
I20260807 20:09:00.104081 14516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.101202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:57759 (local address 127.3.15.129:38131)
0807 20:09:00.101248 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.101251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.101403 (+ 152us) server_negotiation.cc:407] Connection header received
0807 20:09:00.101464 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.101465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.101480 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.101531 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:00.101685 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.102176 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.103202 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.103670 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.103755 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.103763 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.103919 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.103968 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:09:00.104242 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.101075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:38131 (local address 127.3.15.130:57759)
0807 20:09:00.101351 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.101360 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:00.101409 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:00.101532 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:00.101533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:00.101534 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:00.101605 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.101607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.102265 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.102266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.103106 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:00.103109 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.103754 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.103755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:00.103828 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.103832 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:09:00.103912 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:09:00.104091 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":81,"thread_start_us":145,"threads_started":1}
I20260807 20:09:00.104236 14451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c8a919582c4e528c5bc891f0c4b331" candidate_uuid: "57cca4fbd0d74d789ae832a899104b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" is_pre_election: true
I20260807 20:09:00.104460 14451 raft_consensus.cc:2454] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57cca4fbd0d74d789ae832a899104b0d in term 0.
I20260807 20:09:00.104585 14169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c8a919582c4e528c5bc891f0c4b331" candidate_uuid: "57cca4fbd0d74d789ae832a899104b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79b8926577c480e8240de46fe9e2a53" is_pre_election: true
I20260807 20:09:00.104780 14169 raft_consensus.cc:2454] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57cca4fbd0d74d789ae832a899104b0d in term 0.
I20260807 20:09:00.104736 14240 leader_election.cc:302] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57cca4fbd0d74d789ae832a899104b0d, 6b774a0ae27b46d885f5c3a7a41ff27b; no voters:
I20260807 20:09:00.104923 14524 raft_consensus.cc:2781] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:09:00.104993 14524 raft_consensus.cc:482] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:09:00.105018 14524 raft_consensus.cc:3037] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:00.105763 14524 raft_consensus.cc:504] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.105907 14524 leader_election.cc:288] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [CANDIDATE]: Term 1 election: Requested vote from peers 6b774a0ae27b46d885f5c3a7a41ff27b (127.3.15.131:44699), c79b8926577c480e8240de46fe9e2a53 (127.3.15.129:38131)
I20260807 20:09:00.106094 14169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c8a919582c4e528c5bc891f0c4b331" candidate_uuid: "57cca4fbd0d74d789ae832a899104b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79b8926577c480e8240de46fe9e2a53"
I20260807 20:09:00.106139 14451 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c8a919582c4e528c5bc891f0c4b331" candidate_uuid: "57cca4fbd0d74d789ae832a899104b0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b"
I20260807 20:09:00.106180 14169 raft_consensus.cc:3037] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:00.106217 14451 raft_consensus.cc:3037] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:00.106954 14451 raft_consensus.cc:2454] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57cca4fbd0d74d789ae832a899104b0d in term 1.
I20260807 20:09:00.107061 14169 raft_consensus.cc:2454] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57cca4fbd0d74d789ae832a899104b0d in term 1.
I20260807 20:09:00.107124 14240 leader_election.cc:302] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57cca4fbd0d74d789ae832a899104b0d, 6b774a0ae27b46d885f5c3a7a41ff27b; no voters:
I20260807 20:09:00.107203 14524 raft_consensus.cc:2781] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:09:00.107353 14524 raft_consensus.cc:686] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 1 LEADER]: Becoming Leader. State: Replica: 57cca4fbd0d74d789ae832a899104b0d, State: Running, Role: LEADER
I20260807 20:09:00.107475 14524 consensus_queue.cc:237] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } }
I20260807 20:09:00.108407 14015 catalog_manager.cc:5495] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 57cca4fbd0d74d789ae832a899104b0d (127.3.15.130). New cstate: current_term: 1 leader_uuid: "57cca4fbd0d74d789ae832a899104b0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57cca4fbd0d74d789ae832a899104b0d" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 34175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 } health_report { overall_health: UNKNOWN } } }
I20260807 20:09:00.122989 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.120542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33480 (local address 127.3.15.130:34175)
0807 20:09:00.120617 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:00.120619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:00.120634 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:09:00.120669 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:00.120670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:00.120689 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:00.120758 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:09:00.121001 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.121505 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.122246 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:00.122429 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:00.122499 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:00.122640 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:09:00.122745 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:09:00.122809 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:00.122841 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 20:09:00.122860 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 20:09:00.124847 14451 raft_consensus.cc:1261] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 1 FOLLOWER]: Refusing update from remote peer 57cca4fbd0d74d789ae832a899104b0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:09:00.124871 14169 raft_consensus.cc:1261] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53 [term 1 FOLLOWER]: Refusing update from remote peer 57cca4fbd0d74d789ae832a899104b0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:09:00.125213 14524 consensus_queue.cc:1035] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b774a0ae27b46d885f5c3a7a41ff27b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 44699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:09:00.125363 14528 consensus_queue.cc:1035] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79b8926577c480e8240de46fe9e2a53" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 38131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:09:00.128949 14532 mvcc.cc:205] Tried to move back new op lower bound from 7316002161148628992 to 7316002161080954880. Current Snapshot: MvccSnapshot[applied={T|T < 7316002161148628992}]
I20260807 20:09:00.130834 14014 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:09:00.130946 14014 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:09:00.131824 14014 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4fcf6e7892fd4ca584a848c67ca1de91: Sending DeleteTablet for 3 replicas of tablet 06c8a919582c4e528c5bc891f0c4b331
I20260807 20:09:00.132225 14148 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c8a919582c4e528c5bc891f0c4b331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:09:00 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:34608
I20260807 20:09:00.132280 14290 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c8a919582c4e528c5bc891f0c4b331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:09:00 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:33466
I20260807 20:09:00.132457 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 14081
I20260807 20:09:00.132498 14540 tablet_replica.cc:330] T 06c8a919582c4e528c5bc891f0c4b331 P c79b8926577c480e8240de46fe9e2a53: stopping tablet replica
I20260807 20:09:00.132582 14431 tablet_service.cc:1514] Processing DeleteTablet for tablet 06c8a919582c4e528c5bc891f0c4b331 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:09:00 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:50466
I20260807 20:09:00.135222 14542 tablet_replica.cc:330] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: stopping tablet replica
I20260807 20:09:00.135373 14542 raft_consensus.cc:2229] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:09:00.135515 14542 raft_consensus.cc:2258] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:09:00.135740 14541 tablet_replica.cc:330] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: stopping tablet replica
I20260807 20:09:00.135862 14541 raft_consensus.cc:2229] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:09:00.135974 14541 raft_consensus.cc:2258] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:09:00.136413 14541 ts_tablet_manager.cc:1905] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:09:00.137140 14542 ts_tablet_manager.cc:1905] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:09:00.138444 14541 ts_tablet_manager.cc:1918] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:09:00.138525 14541 log.cc:1196] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/06c8a919582c4e528c5bc891f0c4b331
I20260807 20:09:00.138865 14541 ts_tablet_manager.cc:1939] T 06c8a919582c4e528c5bc891f0c4b331 P 57cca4fbd0d74d789ae832a899104b0d: Deleting consensus metadata
I20260807 20:09:00.139153 14542 ts_tablet_manager.cc:1918] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:09:00.139288 14542 log.cc:1196] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/06c8a919582c4e528c5bc891f0c4b331
I20260807 20:09:00.139338 14001 catalog_manager.cc:4841] TS 57cca4fbd0d74d789ae832a899104b0d (127.3.15.130:34175): tablet 06c8a919582c4e528c5bc891f0c4b331 (table test-table [id=b266e12acf0c4d978adf7ccfd12d2202]) successfully deleted
I20260807 20:09:00.139667 14542 ts_tablet_manager.cc:1939] T 06c8a919582c4e528c5bc891f0c4b331 P 6b774a0ae27b46d885f5c3a7a41ff27b: Deleting consensus metadata
I20260807 20:09:00.140122 14000 catalog_manager.cc:4841] TS 6b774a0ae27b46d885f5c3a7a41ff27b (127.3.15.131:44699): tablet 06c8a919582c4e528c5bc891f0c4b331 (table test-table [id=b266e12acf0c4d978adf7ccfd12d2202]) successfully deleted
W20260807 20:09:00.141059 14003 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:38131 (error 108)
I20260807 20:09:00.141247 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 14222
I20260807 20:09:00.143702 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:00.143547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:38131 (local address 127.0.0.1:34624)
0807 20:09:00.143611 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:00.143662 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:38131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 20:09:00.143860 14003 catalog_manager.cc:4584] TS c79b8926577c480e8240de46fe9e2a53 (127.3.15.129:38131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06c8a919582c4e528c5bc891f0c4b331: Network error: Client connection negotiation failed: client connection to 127.3.15.129:38131: connect: Connection refused (error 111)
I20260807 20:09:00.147972 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 14362
I20260807 20:09:00.153401 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 13984
2026-08-07T20:09:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2031 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:09:00 dist-test-slave-1x4g krb5kdc[14547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:09:00 dist-test-slave-1x4g krb5kdc[14547](info): set up 2 sockets
Aug 07 20:09:00 dist-test-slave-1x4g krb5kdc[14547](info): commencing operation
krb5kdc: starting...
W20260807 20:09:01.246963 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:09:01 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:09:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:09:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/rangeradmin_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/rangeradmin_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/rangerlookup_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/rangerlookup_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
I20260807 20:09:01.431464 3134 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskp_X5oZ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/postgres -l logfile start
2026-08-07 20:09:04.061 UTC [14578] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 20:09:04.061 UTC [14578] LOG: listening on IPv4 address "127.3.15.148", port 34155
2026-08-07 20:09:04.065 UTC [14578] LOG: listening on Unix socket "/tmp/.s.PGSQL.34155"
2026-08-07 20:09:04.102 UTC [14582] LOG: database system was shut down at 2026-08-07 20:09:02 UTC
2026-08-07 20:09:04.107 UTC [14578] LOG: database system is ready to accept connections
I20260807 20:09:05.083263 3134 mini_postgres.cc:96] Postgres bound to 34155
2026-08-07 20:09:05.092 UTC [14590] FATAL: database "slave" does not exist
127.3.15.148:34155 - accepting connections
I20260807 20:09:05.093000 3134 mini_ranger.cc:161] Starting Ranger...
I20260807 20:09:05.108608 3134 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 20:09:05.607578 3134 mini_ranger.cc:87] Created ranger Postgres database
I20260807 20:09:05.607668 3134 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-admin
2026-08-07 20:09:05,940 [I] DB FLAVOR :POSTGRES
2026-08-07 20:09:05,940 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:09:05,940 [I] Checking connection..
2026-08-07 20:09:05,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 20:09:06,202 [I] Checking connection passed.
2026-08-07 20:09:06,202 [I] --------- Verifying version history table ---------
2026-08-07 20:09:06,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:09:06,463 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:09:06,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:09:06,716 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:09:06,717 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 20:09:06,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 20:09:06.966 UTC [14657] WARNING: there is no transaction in progress
2026-08-07 20:09:06,972 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 20:09:06,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:09:07,236 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 20:09:07,236 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:09:07,236 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 20:09:07,472 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 20:09:07,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-07 20:09:07,941 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:09:08,182 [I] Table x_portal_user does not exist in database ranger
2026-08-07 20:09:08,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:09:08,423 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 20:09:08,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:09:08,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:09:08,936 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 20:09:08,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 20:09:09.922 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:09.938 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:09.955 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:09.970 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:09.990 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.012 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.071 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.078 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.092 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.102 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.112 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.128 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.138 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.146 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.154 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10.162 UTC [14772] WARNING: there is no transaction in progress
2026-08-07 20:09:10,831 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 20:09:10,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 20:09:11,097 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 20:09:11,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:09:11,343 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 20:09:11,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:09:11,611 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 20:09:11,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:09:11,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:09:12,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-07 20:09:12,368 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 20:09:12,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 20:09:12,637 [I] DB_PATCHES have already been applied
I20260807 20:09:12.643275 3134 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-admin:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskp_X5oZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 20:09:12.643378 3134 mini_ranger.cc:193] Using host: 127.3.15.148
I20260807 20:09:12.647385 3134 mini_ranger.cc:227] Ranger admin URL: http://127.3.15.148:43443
Aug 07, 2026 8:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:09:12 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:09:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:09:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:09:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:09:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:09:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 20:09:13.821573 3134 mini_ranger.cc:160] Time spent starting Ranger: real 8.729s user 0.000s sys 0.011s
Aug 07, 2026 8:09:35 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:09:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:09:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:09:35 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:09:36 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 20:09:37.254842 3134 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/rangerkms_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/rangerkms_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.15.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.15.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 20:09:37 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133377, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 20:09:37.346668 3134 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 20:09:37.357270 3134 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 20:09:38.233114 3134 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 20:09:38.233211 3134 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms
I20260807 20:09:38.233235 3134 mini_ranger_kms.cc:227] Using postgres at 127.3.15.148:34155
2026-08-07 20:09:38,508 [I] DB FLAVOR :POSTGRES
2026-08-07 20:09:38,509 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:09:38,509 [I] Checking connection
2026-08-07 20:09:38,509 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 20:09:38,766 [I] connection success
2026-08-07 20:09:38,766 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 20:09:38,766 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 20:09:38,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 20:09:39,046 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 20:09:39,046 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:09:39,046 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 20:09:39,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:34155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 20:09:39,315 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 20:09:39.831972 3134 mini_ranger_kms.cc:310] Created kms service
I20260807 20:09:40.003540 3134 mini_ranger_kms.cc:326] Created kudu user
I20260807 20:09:40.044673 3134 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 20:09:40 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133377, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:09:40.426805 3134 mini_ranger_kms.cc:384] Added ranger policy
I20260807 20:09:40.426968 3134 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms:/tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 20:09:40.426997 3134 mini_ranger_kms.cc:240] Using host: 127.3.15.148
I20260807 20:09:40.432631 3134 mini_ranger_kms.cc:276] Ranger KMS PID: 14961
I20260807 20:09:40.432814 3134 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.15.148:40835
Aug 07, 2026 8:09:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:09:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:09:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:09:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:09:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 20:09:42 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133382, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.15.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:09:42.849929 3134 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.503s user 0.000s sys 0.255s
I20260807 20:09:42.850111 3134 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 20:09:42.850239 3134 mini_ranger_kms.cc:401] 127.3.15.148:40835/kms/v1/keys
Aug 07 20:09:43 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133382, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.15.148@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
Aug 07 20:09:43 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133382, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.15.148@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
Aug 07 20:09:45 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:09:46.025198 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44525
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:44525
--ranger_config_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:40835/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:09:46.125914 15011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:09:46.126173 15011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:09:46.126214 15011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:09:46.127470 15011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:09:46.127543 15011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:09:46.127570 15011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:09:46.127595 15011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:09:46.127625 15011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:09:46.129352 15011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.15.148:40835/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:44525
--ranger_config_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:09:46.129644 15011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:09:46.129916 15011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:09:46.136745 15011 server_base.cc:1034] running on GCE node
W20260807 20:09:46.136710 15018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:46.136834 15017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:46.136974 15020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:09:46.137257 15011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:09:46.137585 15011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:09:46.138777 15011 hybrid_clock.cc:648] HybridClock initialized: now 1786133386138756 us; error 35 us; skew 500 ppm
Aug 07 20:09:46 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133386, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:09:46.145306 15011 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:09:46.145886 15011 webserver.cc:460] Webserver started at http://127.3.15.190:46651/ using document root <none> and password file <none>
I20260807 20:09:46.146168 15011 fs_manager.cc:359] Metadata directory not provided
I20260807 20:09:46.146215 15011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:09:46.146358 15011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:09:46 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133386, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:09:46.209769 15011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "f0bbf4ca29dc421182b1eccc60a6d9db"
format_stamp: "Formatted at 2026-08-07 20:09:46 on dist-test-slave-1x4g"
server_key: "cd5cbf367efb2d8fe20b4ab0998f430f"
server_key_iv: "39e05b895c2453f982c30632a8c87fdf"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:46.210255 15011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "f0bbf4ca29dc421182b1eccc60a6d9db"
format_stamp: "Formatted at 2026-08-07 20:09:46 on dist-test-slave-1x4g"
server_key: "cd5cbf367efb2d8fe20b4ab0998f430f"
server_key_iv: "39e05b895c2453f982c30632a8c87fdf"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:46.212343 15011 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 20:09:46.228009 15028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:46.228296 15011 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:09:46.228470 15011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "f0bbf4ca29dc421182b1eccc60a6d9db"
format_stamp: "Formatted at 2026-08-07 20:09:46 on dist-test-slave-1x4g"
server_key: "cd5cbf367efb2d8fe20b4ab0998f430f"
server_key_iv: "39e05b895c2453f982c30632a8c87fdf"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:46.228626 15011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:09:46.241336 15011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:09:46.243929 15011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:09:46.244153 15011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:09:46.253993 15011 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:44525
I20260807 20:09:46.254069 15082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:44525 every 8 connection(s)
I20260807 20:09:46.254568 15011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:09:46.255712 15083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:46.259099 15083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db: Bootstrap starting.
I20260807 20:09:46.260416 15083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:46.260957 15083 log.cc:824] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:46.261843 15083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db: No bootstrap required, opened a new log
I20260807 20:09:46.263167 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 15011
I20260807 20:09:46.263281 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:09:46.263481 3134 external_mini_cluster.cc:1428] Setting key e776951c54d107a5c821609ab3a56925
I20260807 20:09:46.265352 15083 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44525 } }
I20260807 20:09:46.265535 15083 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:46.265599 15083 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0bbf4ca29dc421182b1eccc60a6d9db, State: Initialized, Role: FOLLOWER
I20260807 20:09:46.265776 15083 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44525 } }
I20260807 20:09:46.265918 15083 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:09:46.265955 15083 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:09:46.266037 15083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:46.266954 15083 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44525 } }
I20260807 20:09:46.267086 15083 leader_election.cc:302] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0bbf4ca29dc421182b1eccc60a6d9db; no voters:
I20260807 20:09:46.267307 15083 leader_election.cc:288] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:09:46.267436 15088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:09:46.267692 15088 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [term 1 LEADER]: Becoming Leader. State: Replica: f0bbf4ca29dc421182b1eccc60a6d9db, State: Running, Role: LEADER
I20260807 20:09:46.267800 15088 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44525 } }
I20260807 20:09:46.268198 15088 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0bbf4ca29dc421182b1eccc60a6d9db. Latest consensus state: current_term: 1 leader_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44525 } } }
I20260807 20:09:46.268291 15088 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [sys.catalog]: This master's current role is: LEADER
I20260807 20:09:46.270426 15089 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0bbf4ca29dc421182b1eccc60a6d9db" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44525 } } }
I20260807 20:09:46.270510 15089 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [sys.catalog]: This master's current role is: LEADER
I20260807 20:09:46.270772 15083 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:09:46.270949 15091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:09:46.271605 15091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:09:46.272459 15083 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 07 20:09:46 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:09:46.280643 15091 catalog_manager.cc:1347] Generated new cluster ID: 308d92f247894457bfecc4d754da7b94
I20260807 20:09:46.280992 15091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:09:46.284070 15087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:46.264124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59416 (local address 127.3.15.190:44525)
0807 20:09:46.264319 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:46.264340 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:46.264442 (+ 102us) server_negotiation.cc:407] Connection header received
0807 20:09:46.265348 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:46.265360 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:46.265446 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:46.265616 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:46.266683 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:46.270044 (+ 3361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:46.271224 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:46.271858 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:46.271966 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:46.275835 (+ 3869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:46.275938 (+ 103us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:46.275938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:46.276188 (+ 250us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:46.280994 (+ 4806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:46.281502 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:46.281505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:46.281505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:46.281699 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:46.281907 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:46.281908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:46.281908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:46.282323 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:46.283066 (+ 743us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:46.283139 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 20:09:46.283450 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260807 20:09:46.311044 15091 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:09:46.312227 15091 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:09:46.321116 15091 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db: Generated new TSK 0
I20260807 20:09:46.321396 15091 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:09:47.119481 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:40835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44525
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:09:47.237571 15128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:09:47.237823 15128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:09:47.237866 15128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:09:47.239120 15128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:09:47.239188 15128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:09:47.239233 15128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:09:47.240911 15128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.15.148:40835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:09:47.241179 15128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:09:47.241452 15128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:09:47.247622 15134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:47.247565 15136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:47.247951 15133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:09:47.248862 15128 server_base.cc:1034] running on GCE node
I20260807 20:09:47.249089 15128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:09:47.249364 15128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:09:47.250556 15128 hybrid_clock.cc:648] HybridClock initialized: now 1786133387250540 us; error 49 us; skew 500 ppm
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:09:47.253383 15128 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:09:47.254091 15128 webserver.cc:460] Webserver started at http://127.3.15.129:44173/ using document root <none> and password file <none>
I20260807 20:09:47.254364 15128 fs_manager.cc:359] Metadata directory not provided
I20260807 20:09:47.254462 15128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:09:47.254642 15128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:09:47.270305 15128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "156acac96ee245e196d021c9c8f13f93"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "2b5ba74c15d195e4aa3f046c8a74adcb"
server_key_iv: "c533953419dc7dfd5a4608ff426637f2"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.270990 15128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "156acac96ee245e196d021c9c8f13f93"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "2b5ba74c15d195e4aa3f046c8a74adcb"
server_key_iv: "c533953419dc7dfd5a4608ff426637f2"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.273523 15128 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 20:09:47.292956 15143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.293396 15128 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260807 20:09:47.293530 15128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "156acac96ee245e196d021c9c8f13f93"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "2b5ba74c15d195e4aa3f046c8a74adcb"
server_key_iv: "c533953419dc7dfd5a4608ff426637f2"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.293666 15128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:09:47.317838 15128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:09:47.320950 15128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:09:47.321141 15128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:09:47.321583 15128 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:09:47.322080 15128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:09:47.322135 15128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.322167 15128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:09:47.322221 15128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.335125 15128 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:36675
I20260807 20:09:47.335335 15260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:36675 every 8 connection(s)
I20260807 20:09:47.335733 15128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:09:47.337057 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 15128
I20260807 20:09:47.337163 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:09:47.337379 3134 external_mini_cluster.cc:1428] Setting key 01718d663ffbbfce80152e46a05e87e1
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:09:47.353160 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.336906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:35801 (local address 127.3.15.190:44525)
0807 20:09:47.337047 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.337049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.338340 (+ 1291us) server_negotiation.cc:407] Connection header received
0807 20:09:47.339493 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.339495 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.339509 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.339550 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:47.340911 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.341721 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.344871 (+ 3150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.345185 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.345238 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.349447 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:47.349449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:47.349449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:47.349460 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:47.351501 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.351970 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.351972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.351972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.352056 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.352320 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.352321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.352321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.352483 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:47.352555 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.352865 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.353024 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260807 20:09:47.353298 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.337622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44525 (local address 127.3.15.129:35801)
0807 20:09:47.338151 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:47.338203 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:47.339356 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:47.339791 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:47.339795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:47.340319 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:09:47.340751 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.340760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.341839 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.341840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.343270 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.343274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.345351 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.345352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.345609 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.346710 (+ 1101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:09:47.346723 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:09:47.349299 (+ 2576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:09:47.351598 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:47.351605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:47.351607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:47.351846 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:47.352133 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:47.352134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:47.352134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:47.352240 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:47.352550 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:09:47.352555 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:09:47.352687 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 20:09:47.352767 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:09:47.353061 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":167,"threads_started":1}
I20260807 20:09:47.353955 15261 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44525
I20260807 20:09:47.354068 15261 heartbeater.cc:464] Registering TS with master...
I20260807 20:09:47.354270 15261 heartbeater.cc:511] Master 127.3.15.190:44525 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
I20260807 20:09:47.354885 15046 ts_manager.cc:194] Registered new tserver with Master: 156acac96ee245e196d021c9c8f13f93 (127.3.15.129:36675)
I20260807 20:09:47.355746 15046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:35801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:09:47.377694 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:40835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44525
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:09:47.523229 15269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:09:47.523527 15269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:09:47.523566 15269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:09:47.525363 15269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:09:47.525421 15269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:09:47.525462 15269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:09:47.527662 15269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.15.148:40835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:09:47.528076 15269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:09:47.528370 15269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:09:47.535951 15279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:47.536155 15277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:47.536201 15276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:09:47.537313 15269 server_base.cc:1034] running on GCE node
I20260807 20:09:47.537545 15269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:09:47.537793 15269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:09:47.538964 15269 hybrid_clock.cc:648] HybridClock initialized: now 1786133387538928 us; error 47 us; skew 500 ppm
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:09:47.543221 15269 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:09:47.543910 15269 webserver.cc:460] Webserver started at http://127.3.15.130:44309/ using document root <none> and password file <none>
I20260807 20:09:47.544253 15269 fs_manager.cc:359] Metadata directory not provided
I20260807 20:09:47.544338 15269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:09:47.544512 15269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:09:47.557732 15269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "4cb68cf5c5ae414fae606f04a98ee582"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "69fa04b7c139750da856c36bde571b0d"
server_key_iv: "b99675b66ffe4591548094d1798ec91b"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.558251 15269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "4cb68cf5c5ae414fae606f04a98ee582"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "69fa04b7c139750da856c36bde571b0d"
server_key_iv: "b99675b66ffe4591548094d1798ec91b"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.572178 15269 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.002s sys 0.001s
I20260807 20:09:47.592983 15288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.593251 15269 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260807 20:09:47.593385 15269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "4cb68cf5c5ae414fae606f04a98ee582"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "69fa04b7c139750da856c36bde571b0d"
server_key_iv: "b99675b66ffe4591548094d1798ec91b"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.593559 15269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:09:47.621332 15269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:09:47.623468 15269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:09:47.623665 15269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:09:47.623922 15269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:09:47.624341 15269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:09:47.624379 15269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.624402 15269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:09:47.624450 15269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.637426 15269 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:37523
I20260807 20:09:47.637944 15269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:09:47.640902 15405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:37523 every 8 connection(s)
I20260807 20:09:47.645612 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 15269
I20260807 20:09:47.645730 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:09:47.645975 3134 external_mini_cluster.cc:1428] Setting key 43d02e9deb135f27827ce941f47d3127
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:09:47.658278 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.639387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:59175 (local address 127.3.15.190:44525)
0807 20:09:47.639468 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.639471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.639618 (+ 147us) server_negotiation.cc:407] Connection header received
0807 20:09:47.640590 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.640591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.640614 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.640672 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:47.641932 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.642758 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.650264 (+ 7506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.652211 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.652303 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.655233 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:47.655235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:47.655235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:47.655245 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:47.656812 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.657259 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.657260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.657261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.657320 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.657590 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.657591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.657591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.657709 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:47.657781 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.658077 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.658184 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 20:09:47.658609 15408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.639003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44525 (local address 127.3.15.130:59175)
0807 20:09:47.639313 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:47.639471 (+ 158us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:47.640432 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:47.640771 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:47.640776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:47.641334 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:09:47.641806 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.641814 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.645077 (+ 3263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.645079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.650150 (+ 5071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.650154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.652337 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.652338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.652453 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.653210 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:09:47.653221 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:09:47.655109 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:09:47.656918 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:47.656926 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:47.656928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:47.657171 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:47.657389 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:47.657390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:47.657391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:47.657515 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:47.657785 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:09:47.657792 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:09:47.657939 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 20:09:47.658044 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 20:09:47.658201 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":154,"threads_started":1}
I20260807 20:09:47.659080 15406 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44525
I20260807 20:09:47.659214 15406 heartbeater.cc:464] Registering TS with master...
I20260807 20:09:47.659443 15406 heartbeater.cc:511] Master 127.3.15.190:44525 requested a full tablet report, sending...
I20260807 20:09:47.659853 15046 ts_manager.cc:194] Registered new tserver with Master: 4cb68cf5c5ae414fae606f04a98ee582 (127.3.15.130:37523)
I20260807 20:09:47.660249 15046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:59175
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:09:47.688213 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:40835/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44525
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:09:47.830143 15413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:09:47.830427 15413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:09:47.830466 15413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:09:47.832255 15413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:09:47.832405 15413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:09:47.832491 15413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:09:47.834663 15413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.15.148:40835/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:09:47.834972 15413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:09:47.835224 15413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:09:47.842645 15419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:09:47.843493 15418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:09:47.843602 15413 server_base.cc:1034] running on GCE node
W20260807 20:09:47.843791 15422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:09:47.844022 15413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:09:47.844331 15413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:09:47.845525 15413 hybrid_clock.cc:648] HybridClock initialized: now 1786133387845494 us; error 46 us; skew 500 ppm
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:09:47.848903 15413 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:09:47.849612 15413 webserver.cc:460] Webserver started at http://127.3.15.131:34215/ using document root <none> and password file <none>
I20260807 20:09:47.849879 15413 fs_manager.cc:359] Metadata directory not provided
I20260807 20:09:47.849956 15413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:09:47.850099 15413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:09:47.863374 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "0c5aa79fe9e64357ae6261b489910e84"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "ccb3f6a5f190e8eaa6505fb082d4619a"
server_key_iv: "a2bd72edd77eb1db92b99e9a50e31eeb"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.863843 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "0c5aa79fe9e64357ae6261b489910e84"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "ccb3f6a5f190e8eaa6505fb082d4619a"
server_key_iv: "a2bd72edd77eb1db92b99e9a50e31eeb"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.867089 15413 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 20:09:47.890967 15429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.891250 15413 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.000s
I20260807 20:09:47.891366 15413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "0c5aa79fe9e64357ae6261b489910e84"
format_stamp: "Formatted at 2026-08-07 20:09:47 on dist-test-slave-1x4g"
server_key: "ccb3f6a5f190e8eaa6505fb082d4619a"
server_key_iv: "a2bd72edd77eb1db92b99e9a50e31eeb"
server_key_version: "kuduclusterkey@0"
I20260807 20:09:47.891499 15413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:09:47.910152 15413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:09:47.912581 15413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:09:47.912820 15413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:09:47.913292 15413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:09:47.913728 15413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:09:47.913789 15413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.913833 15413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:09:47.913861 15413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:09:47.931155 15413 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:34823
I20260807 20:09:47.931838 15413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:09:47.937801 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 15413
I20260807 20:09:47.938073 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:09:47.938333 3134 external_mini_cluster.cc:1428] Setting key e699dc8fdbbac2c08c7a759aa8fe4bb0
I20260807 20:09:47.937707 15548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:34823 every 8 connection(s)
Aug 07 20:09:47 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:09:47.949214 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.938885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:57485 (local address 127.3.15.190:44525)
0807 20:09:47.939056 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.939058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.939821 (+ 763us) server_negotiation.cc:407] Connection header received
0807 20:09:47.940330 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.940331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.940343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.940414 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:47.941174 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.941782 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.943221 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.943444 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.943520 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.945797 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:47.945799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:47.945799 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:47.945807 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:47.947565 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.948045 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.948046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.948047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.948108 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.948386 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.948400 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.948401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.948553 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:47.948601 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.949024 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.949113 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 20:09:47.950711 15551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.938677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44525 (local address 127.3.15.131:57485)
0807 20:09:47.939647 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:47.939701 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:47.940229 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:47.940492 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:47.940496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:47.940703 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:09:47.941024 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.941033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.941907 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.941908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.943128 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.943131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.943540 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.943541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.943663 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.944219 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:09:47.944228 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:09:47.945700 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:09:47.947660 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:47.947667 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:47.947669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:47.947954 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:47.948180 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:09:47.948181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:09:47.948181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:09:47.948307 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:09:47.948652 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:09:47.948659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:09:47.948848 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 20:09:47.949623 (+ 775us) client_negotiation.cc:240] Negotiation successful
0807 20:09:47.950466 (+ 843us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":413,"threads_started":1}
I20260807 20:09:47.951599 15549 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44525
I20260807 20:09:47.951750 15549 heartbeater.cc:464] Registering TS with master...
I20260807 20:09:47.952004 15549 heartbeater.cc:511] Master 127.3.15.190:44525 requested a full tablet report, sending...
I20260807 20:09:47.952414 15046 ts_manager.cc:194] Registered new tserver with Master: 0c5aa79fe9e64357ae6261b489910e84 (127.3.15.131:34823)
I20260807 20:09:47.952853 15046 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:57485
I20260807 20:09:47.961524 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:09:47.971890 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.963612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.3.15.190:44525)
0807 20:09:47.963661 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.963663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.964087 (+ 424us) server_negotiation.cc:407] Connection header received
0807 20:09:47.964202 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.964204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.964217 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.964254 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:09:47.965086 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.965759 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.966573 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.966781 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.966883 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.968564 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:09:47.968565 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:09:47.968566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:09:47.968574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:09:47.970475 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.970931 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.970932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.970933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.971000 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:09:47.971207 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:09:47.971208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:09:47.971209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:09:47.971409 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:09:47.971541 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.971681 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.971769 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:09:47.973986 15046 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:09:47.976478 15046 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:09:47.988662 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.984227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.3.15.130:37523)
0807 20:09:47.984723 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.984728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.985038 (+ 310us) server_negotiation.cc:407] Connection header received
0807 20:09:47.985121 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.985123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.985138 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.985384 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:47.985590 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.986369 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.987659 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.988190 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.988300 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.988397 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.988522 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.988565 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":230,"threads_started":1}
I20260807 20:09:47.988654 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.983350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:37523 (local address 127.0.0.1:44342)
0807 20:09:47.984730 (+ 1380us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:47.984742 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:47.985036 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:47.985413 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:47.985417 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:47.985420 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:47.985495 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.985498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.986469 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.986471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.987522 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.987525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.988302 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.988304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.988400 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.988418 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 20:09:47.988504 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 20:09:47.988562 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"thread_start_us":174,"threads_started":1}
I20260807 20:09:47.991110 15338 tablet_service.cc:1467] Processing CreateTablet for tablet 4e282b35bbd443b1a0ef3007c8ad36d9 (DEFAULT_TABLE table=test-table [id=7354de7051be4deab1adf38a81e2e6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:09:47.991533 15338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e282b35bbd443b1a0ef3007c8ad36d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:47.993103 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.983835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.3.15.131:34823)
0807 20:09:47.984072 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.984078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.988946 (+ 4868us) server_negotiation.cc:407] Connection header received
0807 20:09:47.989091 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.989093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.989112 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.989333 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:47.989601 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.990334 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.991786 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.992547 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.992603 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.992680 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.992964 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.993011 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260807 20:09:47.993430 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.983708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:34823 (local address 127.0.0.1:52096)
0807 20:09:47.988883 (+ 5175us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:47.988894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:47.988979 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:47.989486 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:47.989487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:47.989489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:47.989520 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.989521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.990640 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.990641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.991643 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.991647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.992748 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.992749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.992843 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.992848 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:09:47.993245 (+ 397us) client_negotiation.cc:240] Negotiation successful
0807 20:09:47.993351 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5163,"mutex_wait_us":24,"thread_start_us":350,"threads_started":1}
I20260807 20:09:47.994385 15480 tablet_service.cc:1467] Processing CreateTablet for tablet 4e282b35bbd443b1a0ef3007c8ad36d9 (DEFAULT_TABLE table=test-table [id=7354de7051be4deab1adf38a81e2e6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:09:47.994779 15480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e282b35bbd443b1a0ef3007c8ad36d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:47.997332 15565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.983215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:36675 (local address 127.0.0.1:57416)
0807 20:09:47.984154 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:47.984201 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:47.984483 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:47.993244 (+ 8761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:47.993245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:47.993248 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:47.993343 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.993345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.994717 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.994718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.995873 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:47.995876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.997113 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.997115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:47.997182 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.997187 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:09:47.997218 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 20:09:47.997256 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"mutex_wait_us":24,"thread_start_us":196,"threads_started":1}
I20260807 20:09:47.997871 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:47.983661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57416 (local address 127.3.15.129:36675)
0807 20:09:47.992880 (+ 9219us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:47.992885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:47.992898 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:09:47.992936 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:47.992938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:47.992956 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:47.993413 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:47.993494 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.994563 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.996056 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:47.997007 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:47.997563 (+ 556us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:47.997639 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:47.997668 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 20:09:47.997709 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9176,"thread_start_us":57,"threads_started":1}
I20260807 20:09:47.998759 15186 tablet_service.cc:1467] Processing CreateTablet for tablet 4e282b35bbd443b1a0ef3007c8ad36d9 (DEFAULT_TABLE table=test-table [id=7354de7051be4deab1adf38a81e2e6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:09:47.999205 15186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e282b35bbd443b1a0ef3007c8ad36d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:09:48.000170 15568 tablet_bootstrap.cc:492] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Bootstrap starting.
I20260807 20:09:48.001716 15568 tablet_bootstrap.cc:654] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:48.002239 15568 log.cc:824] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:48.002800 15567 tablet_bootstrap.cc:492] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: Bootstrap starting.
I20260807 20:09:48.004483 15567 tablet_bootstrap.cc:654] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:48.005205 15567 log.cc:824] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:48.005540 15570 tablet_bootstrap.cc:492] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: Bootstrap starting.
I20260807 20:09:48.006341 15568 tablet_bootstrap.cc:492] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: No bootstrap required, opened a new log
I20260807 20:09:48.006474 15568 ts_tablet_manager.cc:1397] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260807 20:09:48.007076 15570 tablet_bootstrap.cc:654] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: Neither blocks nor log segments found. Creating new log.
I20260807 20:09:48.007943 15570 log.cc:824] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: Log is configured to *not* fsync() on all Append() calls
I20260807 20:09:48.009379 15570 tablet_bootstrap.cc:492] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: No bootstrap required, opened a new log
I20260807 20:09:48.009513 15570 ts_tablet_manager.cc:1397] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260807 20:09:48.010461 15568 raft_consensus.cc:348] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.010676 15568 raft_consensus.cc:374] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:48.010725 15568 raft_consensus.cc:729] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c5aa79fe9e64357ae6261b489910e84, State: Initialized, Role: FOLLOWER
I20260807 20:09:48.010905 15568 consensus_queue.cc:260] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.011458 15568 ts_tablet_manager.cc:1428] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 20:09:48.011616 15549 heartbeater.cc:503] Master 127.3.15.190:44525 was elected leader, sending a full tablet report...
I20260807 20:09:48.014895 15567 tablet_bootstrap.cc:492] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: No bootstrap required, opened a new log
I20260807 20:09:48.015316 15567 ts_tablet_manager.cc:1397] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260807 20:09:48.018900 15567 raft_consensus.cc:348] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.019258 15567 raft_consensus.cc:374] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:48.019322 15567 raft_consensus.cc:729] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cb68cf5c5ae414fae606f04a98ee582, State: Initialized, Role: FOLLOWER
I20260807 20:09:48.019505 15567 consensus_queue.cc:260] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.019999 15567 ts_tablet_manager.cc:1428] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260807 20:09:48.020147 15406 heartbeater.cc:503] Master 127.3.15.190:44525 was elected leader, sending a full tablet report...
I20260807 20:09:48.020148 15570 raft_consensus.cc:348] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.020426 15570 raft_consensus.cc:374] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:09:48.020527 15570 raft_consensus.cc:729] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 156acac96ee245e196d021c9c8f13f93, State: Initialized, Role: FOLLOWER
I20260807 20:09:48.021144 15570 consensus_queue.cc:260] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.021709 15570 ts_tablet_manager.cc:1428] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260807 20:09:48.021904 15261 heartbeater.cc:503] Master 127.3.15.190:44525 was elected leader, sending a full tablet report...
W20260807 20:09:48.087263 15262 tablet.cc:2369] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 20:09:48 dist-test-slave-1x4g krb5kdc[14547](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133387, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
W20260807 20:09:48.139369 15407 tablet.cc:2369] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:09:48.144228 15573 raft_consensus.cc:482] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:09:48.144366 15573 raft_consensus.cc:504] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.144951 15573 leader_election.cc:288] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cb68cf5c5ae414fae606f04a98ee582 (127.3.15.130:37523), 156acac96ee245e196d021c9c8f13f93 (127.3.15.129:36675)
I20260807 20:09:48.149567 15551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:48.145219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:36675 (local address 127.3.15.131:45259)
0807 20:09:48.145266 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:48.145281 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:48.145334 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:48.145777 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:48.145786 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:48.145788 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:48.145963 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:48.145966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.146916 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.146917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:48.148173 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:48.148176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.149263 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.149264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:48.149395 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:48.149400 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:09:48.149452 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 20:09:48.149490 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":6}
I20260807 20:09:48.149981 15563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:48.145378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:45259 (local address 127.3.15.129:36675)
0807 20:09:48.145440 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:48.145443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:48.145457 (+ 14us) server_negotiation.cc:407] Connection header received
0807 20:09:48.145503 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:48.145505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:48.145519 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:48.145597 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:48.146066 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.146831 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.148364 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.149162 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.149224 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:48.149233 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:48.149819 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 20:09:48.149866 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:09:48.150364 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e282b35bbd443b1a0ef3007c8ad36d9" candidate_uuid: "0c5aa79fe9e64357ae6261b489910e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156acac96ee245e196d021c9c8f13f93" is_pre_election: true
I20260807 20:09:48.150588 15214 raft_consensus.cc:2454] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c5aa79fe9e64357ae6261b489910e84 in term 0.
I20260807 20:09:48.150940 15431 leader_election.cc:302] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5aa79fe9e64357ae6261b489910e84, 156acac96ee245e196d021c9c8f13f93; no voters:
I20260807 20:09:48.151067 15573 raft_consensus.cc:2781] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:09:48.151118 15573 raft_consensus.cc:482] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:09:48.151139 15573 raft_consensus.cc:3037] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:48.152455 15573 raft_consensus.cc:504] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.152587 15573 leader_election.cc:288] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [CANDIDATE]: Term 1 election: Requested vote from peers 4cb68cf5c5ae414fae606f04a98ee582 (127.3.15.130:37523), 156acac96ee245e196d021c9c8f13f93 (127.3.15.129:36675)
I20260807 20:09:48.152876 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e282b35bbd443b1a0ef3007c8ad36d9" candidate_uuid: "0c5aa79fe9e64357ae6261b489910e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "156acac96ee245e196d021c9c8f13f93"
I20260807 20:09:48.152952 15214 raft_consensus.cc:3037] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:48.154161 15214 raft_consensus.cc:2454] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c5aa79fe9e64357ae6261b489910e84 in term 1.
I20260807 20:09:48.154382 15431 leader_election.cc:302] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c5aa79fe9e64357ae6261b489910e84, 156acac96ee245e196d021c9c8f13f93; no voters:
I20260807 20:09:48.154493 15573 raft_consensus.cc:2781] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:09:48.154910 15573 raft_consensus.cc:686] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 1 LEADER]: Becoming Leader. State: Replica: 0c5aa79fe9e64357ae6261b489910e84, State: Running, Role: LEADER
I20260807 20:09:48.155014 15573 consensus_queue.cc:237] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } }
I20260807 20:09:48.155900 15566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:48.145709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:55737 (local address 127.3.15.130:37523)
0807 20:09:48.146001 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:48.146003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:48.146598 (+ 595us) server_negotiation.cc:407] Connection header received
0807 20:09:48.146697 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:48.146697 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:48.146710 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:48.146739 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:09:48.146953 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.147686 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.149151 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.149739 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.149790 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:48.149799 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:48.155783 (+ 5984us) server_negotiation.cc:299] Negotiation successful
0807 20:09:48.155836 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260807 20:09:48.155944 15046 catalog_manager.cc:5495] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c5aa79fe9e64357ae6261b489910e84 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "0c5aa79fe9e64357ae6261b489910e84" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c5aa79fe9e64357ae6261b489910e84" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34823 } health_report { overall_health: HEALTHY } } }
I20260807 20:09:48.156401 15576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:48.145571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:37523 (local address 127.3.15.131:55737)
0807 20:09:48.146422 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:48.146432 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:48.146626 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:48.146795 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:09:48.146796 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:09:48.146797 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:09:48.146876 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:48.146878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.147805 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.147806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:48.149005 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:09:48.149009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.149977 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.149978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:09:48.155670 (+ 5692us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:48.155677 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:09:48.156140 (+ 463us) client_negotiation.cc:240] Negotiation successful
0807 20:09:48.156190 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":276,"threads_started":1}
I20260807 20:09:48.156953 15357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e282b35bbd443b1a0ef3007c8ad36d9" candidate_uuid: "0c5aa79fe9e64357ae6261b489910e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb68cf5c5ae414fae606f04a98ee582"
I20260807 20:09:48.157085 15357 raft_consensus.cc:3037] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:09:48.158336 15357 raft_consensus.cc:2454] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c5aa79fe9e64357ae6261b489910e84 in term 1.
I20260807 20:09:48.158838 15358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e282b35bbd443b1a0ef3007c8ad36d9" candidate_uuid: "0c5aa79fe9e64357ae6261b489910e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cb68cf5c5ae414fae606f04a98ee582" is_pre_election: true
I20260807 20:09:48.158923 15358 raft_consensus.cc:2362] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0c5aa79fe9e64357ae6261b489910e84 in term 1. Re-sending same reply.
I20260807 20:09:48.187016 15564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:48.184436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.3.15.131:34823)
0807 20:09:48.184566 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 20:09:48.184568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:09:48.184574 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:09:48.184619 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:09:48.184620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:09:48.184639 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:09:48.184719 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:09:48.184905 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.185405 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.186174 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:09:48.186393 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:09:48.186474 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:09:48.186660 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:09:48.186782 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:09:48.186882 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 20:09:48.186901 (+ 19us) server_negotiation.cc:299] Negotiation successful
0807 20:09:48.186921 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 20:09:48.189419 15358 raft_consensus.cc:1261] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 1 FOLLOWER]: Refusing update from remote peer 0c5aa79fe9e64357ae6261b489910e84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:09:48.189419 15214 raft_consensus.cc:1261] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 156acac96ee245e196d021c9c8f13f93 [term 1 FOLLOWER]: Refusing update from remote peer 0c5aa79fe9e64357ae6261b489910e84: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260807 20:09:48.189751 15550 tablet.cc:2369] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:09:48.190165 15578 consensus_queue.cc:1035] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cb68cf5c5ae414fae606f04a98ee582" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:09:48.190337 15573 consensus_queue.cc:1035] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [LEADER]: Connected to new peer: Peer: permanent_uuid: "156acac96ee245e196d021c9c8f13f93" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:09:48.194453 15583 mvcc.cc:205] Tried to move back new op lower bound from 7316002358020333568 to 7316002357883535360. Current Snapshot: MvccSnapshot[applied={T|T < 7316002358020333568}]
I20260807 20:09:48.194617 15581 mvcc.cc:205] Tried to move back new op lower bound from 7316002358020333568 to 7316002357883535360. Current Snapshot: MvccSnapshot[applied={T|T < 7316002358020333568}]
I20260807 20:09:48.198132 15044 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:09:48.198236 15044 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:09:48.199267 15044 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0bbf4ca29dc421182b1eccc60a6d9db: Sending DeleteTablet for 3 replicas of tablet 4e282b35bbd443b1a0ef3007c8ad36d9
I20260807 20:09:48.199730 15338 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e282b35bbd443b1a0ef3007c8ad36d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:09:48 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:44342
I20260807 20:09:48.200449 15186 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e282b35bbd443b1a0ef3007c8ad36d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:09:48 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:57416
I20260807 20:09:48.201143 15480 tablet_service.cc:1514] Processing DeleteTablet for tablet 4e282b35bbd443b1a0ef3007c8ad36d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:09:48 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:52096
I20260807 20:09:48.201969 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 15128
I20260807 20:09:48.204161 15589 tablet_replica.cc:330] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582: stopping tablet replica
I20260807 20:09:48.204285 15589 raft_consensus.cc:2229] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:09:48.204404 15589 raft_consensus.cc:2258] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 4cb68cf5c5ae414fae606f04a98ee582 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 20:09:48.213452 15030 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:36675 (error 108)
I20260807 20:09:48.214514 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 15269
I20260807 20:09:48.217171 15592 tablet_replica.cc:330] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: stopping tablet replica
I20260807 20:09:48.217337 15592 raft_consensus.cc:2229] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:09:48.217504 15592 raft_consensus.cc:2258] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:09:48.227353 15562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:09:48.227175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:36675 (local address 127.0.0.1:57422)
0807 20:09:48.227238 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:48.227298 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:36675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 20:09:48.227626 15030 catalog_manager.cc:4584] TS 156acac96ee245e196d021c9c8f13f93 (127.3.15.129:36675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4e282b35bbd443b1a0ef3007c8ad36d9: Network error: Client connection negotiation failed: client connection to 127.3.15.129:36675: connect: Connection refused (error 111)
I20260807 20:09:48.228163 15592 ts_tablet_manager.cc:1905] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:09:48.230434 15592 ts_tablet_manager.cc:1918] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:09:48.230587 15592 log.cc:1196] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/4e282b35bbd443b1a0ef3007c8ad36d9
I20260807 20:09:48.230978 15592 ts_tablet_manager.cc:1939] T 4e282b35bbd443b1a0ef3007c8ad36d9 P 0c5aa79fe9e64357ae6261b489910e84: Deleting consensus metadata
I20260807 20:09:48.231534 15029 catalog_manager.cc:4841] TS 0c5aa79fe9e64357ae6261b489910e84 (127.3.15.131:34823): tablet 4e282b35bbd443b1a0ef3007c8ad36d9 (table test-table [id=7354de7051be4deab1adf38a81e2e6c4]) successfully deleted
W20260807 20:09:48.232667 15562 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:09:48.232300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:37523 (local address 127.0.0.1:44344)
0807 20:09:48.232344 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:09:48.232355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:09:48.232396 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:09:48.232616 (+ 220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.130:37523: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:09:48.233386 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 15413
I20260807 20:09:48.241894 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 15011
I20260807 20:09:48.248574 3134 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 20:09:48.593075 3134 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 20:09:48.593192 3134 mini_ranger.cc:66] Stopping Ranger...
I20260807 20:09:48.937585 3134 mini_ranger.cc:68] Stopped Ranger
2026-08-07 20:09:48.937 UTC [14578] LOG: received smart shutdown request
2026-08-07 20:09:48.941 UTC [14578] LOG: background worker "logical replication launcher" (PID 14588) exited with exit code 1
2026-08-07 20:09:48.942 UTC [14583] LOG: shutting down
2026-08-07 20:09:49.110 UTC [14578] LOG: database system is shut down
2026-08-07T20:09:49Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49070 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:09:49 dist-test-slave-1x4g krb5kdc[15603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:09:49 dist-test-slave-1x4g krb5kdc[15603](info): set up 2 sockets
Aug 07 20:09:49 dist-test-slave-1x4g krb5kdc[15603](info): commencing operation
krb5kdc: starting...
W20260807 20:09:50.333217 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:09:50 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:09:50Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/rangeradmin_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/rangeradmin_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/rangerlookup_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/rangerlookup_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
I20260807 20:09:50.516048 3134 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskp_X5oZ/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/postgres -l logfile start
2026-08-07 20:09:53.073 UTC [15634] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 20:09:53.073 UTC [15634] LOG: listening on IPv4 address "127.3.15.148", port 56511
2026-08-07 20:09:53.077 UTC [15634] LOG: listening on Unix socket "/tmp/.s.PGSQL.56511"
2026-08-07 20:09:53.098 UTC [15637] LOG: database system was shut down at 2026-08-07 20:09:51 UTC
2026-08-07 20:09:53.102 UTC [15634] LOG: database system is ready to accept connections
I20260807 20:09:54.098944 3134 mini_postgres.cc:96] Postgres bound to 56511
2026-08-07 20:09:54.106 UTC [15645] FATAL: database "slave" does not exist
127.3.15.148:56511 - accepting connections
I20260807 20:09:54.106775 3134 mini_ranger.cc:161] Starting Ranger...
I20260807 20:09:54.123286 3134 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 20:09:54.628574 3134 mini_ranger.cc:87] Created ranger Postgres database
I20260807 20:09:54.628677 3134 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-admin
2026-08-07 20:09:54,920 [I] DB FLAVOR :POSTGRES
2026-08-07 20:09:54,921 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:09:54,921 [I] Checking connection..
2026-08-07 20:09:54,921 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 20:09:55,160 [I] Checking connection passed.
2026-08-07 20:09:55,161 [I] --------- Verifying version history table ---------
2026-08-07 20:09:55,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:09:55,429 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:09:55,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:09:55,692 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 20:09:55,692 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 20:09:55,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 20:09:55.932 UTC [15711] WARNING: there is no transaction in progress
2026-08-07 20:09:55,939 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 20:09:55,939 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 20:09:56,180 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 20:09:56,180 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:09:56,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 20:09:56,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 20:09:56,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1x4g.c.gcp-upstream.internal','N') ;"
2026-08-07 20:09:56,946 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:09:57,186 [I] Table x_portal_user does not exist in database ranger
2026-08-07 20:09:57,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:09:57,424 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 20:09:57,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:09:57,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:09:57,935 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 20:09:57,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 20:09:58.903 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:58.921 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:58.937 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:58.953 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:58.969 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:58.988 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.046 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.054 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.072 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.083 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.094 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.109 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.120 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.128 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.135 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59.141 UTC [15825] WARNING: there is no transaction in progress
2026-08-07 20:09:59,801 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 20:09:59,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 20:10:00,058 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 20:10:00,058 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 20:10:00,314 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 20:10:00,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 20:10:00,557 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 20:10:00,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 20:10:00,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 20:10:01,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1x4g.c.gcp-upstream.internal';"
2026-08-07 20:10:01,323 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 20:10:01,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 20:10:01,572 [I] DB_PATCHES have already been applied
I20260807 20:10:01.578347 3134 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-admin:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskp_X5oZ/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 20:10:01.578456 3134 mini_ranger.cc:193] Using host: 127.3.15.148
I20260807 20:10:01.582602 3134 mini_ranger.cc:227] Ranger admin URL: http://127.3.15.148:51779
Aug 07, 2026 8:10:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:10:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:10:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 8:10:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 8:10:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 8:10:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 8:10:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 20:10:02.734400 3134 mini_ranger.cc:160] Time spent starting Ranger: real 8.628s user 0.000s sys 0.011s
Aug 07, 2026 8:10:22 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 8:10:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 8:10:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 8:10:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 8:10:23 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 20:10:23.983814 3134 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.15.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.15.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/rangerkms_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.15.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/rangerkms_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.15.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.15.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.15.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/HTTP_127.3.15.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 20:10:24 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133424, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 20:10:24.093741 3134 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 20:10:24.105577 3134 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 20:10:25.009837 3134 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 20:10:25.009977 3134 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms
I20260807 20:10:25.009999 3134 mini_ranger_kms.cc:227] Using postgres at 127.3.15.148:56511
2026-08-07 20:10:25,185 [I] DB FLAVOR :POSTGRES
2026-08-07 20:10:25,186 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 20:10:25,186 [I] Checking connection
2026-08-07 20:10:25,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 20:10:25,422 [I] connection success
2026-08-07 20:10:25,422 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 20:10:25,422 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 20:10:25,422 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 20:10:25,658 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 20:10:25,659 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 20:10:25,659 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 20:10:25,659 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.15.148:56511/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 20:10:25,916 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 20:10:26.401090 3134 mini_ranger_kms.cc:310] Created kms service
I20260807 20:10:26.601338 3134 mini_ranger_kms.cc:326] Created kudu user
I20260807 20:10:26.647729 3134 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 20:10:26 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133424, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:27.063964 3134 mini_ranger_kms.cc:384] Added ranger policy
I20260807 20:10:27.064209 3134 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms:/tmp/dist-test-taskp_X5oZ/build/release/bin/postgresql.jar:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskp_X5oZ/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 20:10:27.064244 3134 mini_ranger_kms.cc:240] Using host: 127.3.15.148
I20260807 20:10:27.069033 3134 mini_ranger_kms.cc:276] Ranger KMS PID: 16011
I20260807 20:10:27.069132 3134 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.15.148:51009
Aug 07, 2026 8:10:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 8:10:28 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 8:10:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 8:10:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 8:10:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 8:10:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260807 20:10:29.416594 3134 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.323s user 0.005s sys 0.154s
I20260807 20:10:29.416728 3134 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 20:10:29.416785 3134 mini_ranger_kms.cc:401] 127.3.15.148:51009/kms/v1/keys
Aug 07 20:10:29 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133429, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.15.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 07 20:10:29 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133429, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.15.148@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
Aug 07 20:10:30 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133429, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.15.148@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
Aug 07 20:10:32 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:10:32.650031 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34003
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:34003
--ranger_config_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:32.744171 16063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:32.744406 16063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:32.744442 16063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:32.745625 16063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:32.745692 16063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:32.745716 16063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:32.745747 16063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:32.745774 16063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:32.747519 16063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.15.148:51009/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:34003
--ranger_config_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:32.747800 16063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:32.748015 16063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:32.753944 16069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:32.754030 16068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:32.754241 16071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:32.754261 16063 server_base.cc:1034] running on GCE node
I20260807 20:10:32.754562 16063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:32.754917 16063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:32.756088 16063 hybrid_clock.cc:648] HybridClock initialized: now 1786133432756075 us; error 40 us; skew 500 ppm
Aug 07 20:10:32 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133432, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:32.759500 16063 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:32.760089 16063 webserver.cc:460] Webserver started at http://127.3.15.190:33607/ using document root <none> and password file <none>
I20260807 20:10:32.760349 16063 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:32.760412 16063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:32.760581 16063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:10:32 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133432, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:32.813508 16063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "5e5560dd15cd49ee8cf624df2f7dcf61"
format_stamp: "Formatted at 2026-08-07 20:10:32 on dist-test-slave-1x4g"
server_key: "41fab11432db23e62f34aaf8ad712d9f"
server_key_iv: "e52857de9aa7bbe99fe7225564405b21"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:32.814038 16063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "5e5560dd15cd49ee8cf624df2f7dcf61"
format_stamp: "Formatted at 2026-08-07 20:10:32 on dist-test-slave-1x4g"
server_key: "41fab11432db23e62f34aaf8ad712d9f"
server_key_iv: "e52857de9aa7bbe99fe7225564405b21"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:32.815598 16063 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:10:32.831152 16079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:32.831344 16063 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:10:32.831461 16063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "5e5560dd15cd49ee8cf624df2f7dcf61"
format_stamp: "Formatted at 2026-08-07 20:10:32 on dist-test-slave-1x4g"
server_key: "41fab11432db23e62f34aaf8ad712d9f"
server_key_iv: "e52857de9aa7bbe99fe7225564405b21"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:32.831574 16063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:32.850718 16063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:32.852955 16063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:32.853124 16063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:32.857581 16063 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:34003
I20260807 20:10:32.857622 16135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:34003 every 8 connection(s)
I20260807 20:10:32.858039 16063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:32.858767 16136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:32.861402 16136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Bootstrap starting.
I20260807 20:10:32.862056 16136 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:32.862463 16136 log.cc:824] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:32.863265 16136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: No bootstrap required, opened a new log
I20260807 20:10:32.865417 16136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:32.865576 16136 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:32.865599 16136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5560dd15cd49ee8cf624df2f7dcf61, State: Initialized, Role: FOLLOWER
I20260807 20:10:32.865768 16136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:32.865840 16136 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:10:32.865862 16136 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:10:32.865927 16136 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:32.866477 16136 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:32.866561 16136 leader_election.cc:302] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e5560dd15cd49ee8cf624df2f7dcf61; no voters:
I20260807 20:10:32.866789 16136 leader_election.cc:288] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:10:32.866815 16139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:32.866953 16139 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 LEADER]: Becoming Leader. State: Replica: 5e5560dd15cd49ee8cf624df2f7dcf61, State: Running, Role: LEADER
I20260807 20:10:32.867066 16136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:32.867120 16139 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:32.867502 16139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e5560dd15cd49ee8cf624df2f7dcf61. Latest consensus state: current_term: 1 leader_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } } }
I20260807 20:10:32.867518 16141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } } }
I20260807 20:10:32.867700 16141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:32.867923 16139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:32.868120 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16063
I20260807 20:10:32.868237 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:10:32.868319 16149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:32.868450 3134 external_mini_cluster.cc:1428] Setting key 6bd09b3e18f109cc051e80d2875b07b5
I20260807 20:10:32.868865 16149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:32.869406 16136 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 20:10:32.878098 16149 catalog_manager.cc:1347] Generated new cluster ID: bdb323f6a0954b2d9dacaa01aa84148e
I20260807 20:10:32.878228 16149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:10:32 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:32.884723 16158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:32.869150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.3.15.190:34003)
0807 20:10:32.869536 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:32.869545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:32.869577 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:10:32.869705 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:32.869713 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:32.869772 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:32.869904 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:32.870777 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:32.872173 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:32.874472 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:32.876000 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:32.876156 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:32.879250 (+ 3094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:32.879255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:32.879256 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:32.879272 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:32.882489 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:32.882953 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:32.882958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:32.882959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:32.883083 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:32.883293 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:32.883294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:32.883294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:32.883644 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:32.883778 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:32.884323 (+ 545us) server_negotiation.cc:299] Negotiation successful
0807 20:10:32.884464 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":68,"threads_started":1}
I20260807 20:10:32.912611 16149 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:10:32.913695 16149 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:10:32.922137 16149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Generated new TSK 0
I20260807 20:10:32.922508 16149 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:10:33.702258 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34003
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:33.826992 16183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:33.827318 16183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:33.827392 16183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:33.829092 16183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:33.829200 16183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:33.829277 16183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:10:33.831326 16183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.15.148:51009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:33.831915 16183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:33.832237 16183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:33.838784 16192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:33.838778 16190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:33.838995 16188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:33.840031 16183 server_base.cc:1034] running on GCE node
I20260807 20:10:33.840250 16183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:33.840535 16183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:33.841701 16183 hybrid_clock.cc:648] HybridClock initialized: now 1786133433841677 us; error 37 us; skew 500 ppm
Aug 07 20:10:33 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133433, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:33.845152 16183 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:10:33.845732 16183 webserver.cc:460] Webserver started at http://127.3.15.129:34717/ using document root <none> and password file <none>
I20260807 20:10:33.846053 16183 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:33.846123 16183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:33.846268 16183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:10:33 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133433, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:33.859683 16183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "37b88e7e09af400bb26628ec591741ec"
format_stamp: "Formatted at 2026-08-07 20:10:33 on dist-test-slave-1x4g"
server_key: "97ff0d4ff8695b02658150d2a29d4625"
server_key_iv: "cbbcbc6acb45007deb5409c161b8f189"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:33.860188 16183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "37b88e7e09af400bb26628ec591741ec"
format_stamp: "Formatted at 2026-08-07 20:10:33 on dist-test-slave-1x4g"
server_key: "97ff0d4ff8695b02658150d2a29d4625"
server_key_iv: "cbbcbc6acb45007deb5409c161b8f189"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:33.862825 16183 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 20:10:33.876917 16200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:33.877211 16183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260807 20:10:33.877382 16183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "37b88e7e09af400bb26628ec591741ec"
format_stamp: "Formatted at 2026-08-07 20:10:33 on dist-test-slave-1x4g"
server_key: "97ff0d4ff8695b02658150d2a29d4625"
server_key_iv: "cbbcbc6acb45007deb5409c161b8f189"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:33.877509 16183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:33.899413 16183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:33.902060 16183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:33.902261 16183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:33.902566 16183 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:33.902993 16183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:33.903087 16183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:33.903182 16183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:33.903262 16183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:33.910457 16183 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:43419
I20260807 20:10:33.911062 16183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:10:33.911432 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16183
I20260807 20:10:33.911541 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:10:33.911739 3134 external_mini_cluster.cc:1428] Setting key bdd52765d24371284fab7af888b76c0f
I20260807 20:10:33.913661 16319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:43419 every 8 connection(s)
Aug 07 20:10:33 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133433, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
I20260807 20:10:33.936487 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:33.913200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34003 (local address 127.3.15.129:49705)
0807 20:10:33.919698 (+ 6498us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:33.919808 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:33.920583 (+ 775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:33.921142 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:33.921147 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:33.921449 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:33.921767 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:33.921775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:33.923002 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:33.923003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:33.924324 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:33.924328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:33.924946 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:33.924947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:33.925101 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:33.925774 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:33.925785 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:33.931796 (+ 6011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:33.934481 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:33.934493 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:33.934497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:33.934874 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:33.935128 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:33.935129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:33.935130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:33.935310 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:33.935670 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:33.935678 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:33.935879 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 20:10:33.936003 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 20:10:33.936206 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6428,"thread_start_us":98,"threads_started":1}
I20260807 20:10:33.937028 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:33.914718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:49705 (local address 127.3.15.190:34003)
0807 20:10:33.920843 (+ 6125us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:33.920846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:33.920856 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:10:33.920889 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:33.920890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:33.920905 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:33.920980 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:33.921896 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:33.922881 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:33.924428 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:33.924763 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:33.924853 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:33.931949 (+ 7096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:33.931951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:33.931952 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:33.931962 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:33.934299 (+ 2337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:33.934977 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:33.934978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:33.934978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:33.935049 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:33.935393 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:33.935394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:33.935395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:33.935570 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:33.935629 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:33.936814 (+ 1185us) server_negotiation.cc:299] Negotiation successful
0807 20:10:33.936920 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6090,"thread_start_us":176,"threads_started":1}
I20260807 20:10:33.938131 16320 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34003
I20260807 20:10:33.938241 16320 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:33.938478 16320 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:33.939096 16097 ts_manager.cc:194] Registered new tserver with Master: 37b88e7e09af400bb26628ec591741ec (127.3.15.129:43419)
I20260807 20:10:33.939970 16097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:49705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:10:33.957630 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34003
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 20:10:34.099484 16326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:34.099943 16326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:34.100057 16326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:34.102283 16326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:34.102344 16326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:34.102391 16326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:10:34.105336 16326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.15.148:51009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:34.105762 16326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:34.106103 16326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:34.114015 16334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:34.114032 16333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:34.114494 16336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:34.115186 16326 server_base.cc:1034] running on GCE node
I20260807 20:10:34.115413 16326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:34.115710 16326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:34.118963 16326 hybrid_clock.cc:648] HybridClock initialized: now 1786133434118933 us; error 39 us; skew 500 ppm
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:34.122445 16326 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:10:34.123083 16326 webserver.cc:460] Webserver started at http://127.3.15.130:33723/ using document root <none> and password file <none>
I20260807 20:10:34.123373 16326 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:34.123440 16326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:34.123632 16326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:34.138818 16326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "5c435e6e09d2a28036912463e7be53e6"
server_key_iv: "310a08bb243d20d82cf3b9b00d4261e6"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.139467 16326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "5c435e6e09d2a28036912463e7be53e6"
server_key_iv: "310a08bb243d20d82cf3b9b00d4261e6"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.142789 16326 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260807 20:10:34.157986 16344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.158264 16326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:34.158442 16326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "5c435e6e09d2a28036912463e7be53e6"
server_key_iv: "310a08bb243d20d82cf3b9b00d4261e6"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.158610 16326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:34.193105 16326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:34.195789 16326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:34.196029 16326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:34.196355 16326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:34.196871 16326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:34.196969 16326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.197044 16326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:34.197103 16326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.209823 16326 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:38173
I20260807 20:10:34.210359 16326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:10:34.211795 16460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:38173 every 8 connection(s)
I20260807 20:10:34.213320 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16326
I20260807 20:10:34.213451 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:10:34.213709 3134 external_mini_cluster.cc:1428] Setting key 7669744423f888aa1cbb0e49cd9479cc
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:34.225489 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:34.212983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:50903 (local address 127.3.15.190:34003)
0807 20:10:34.213033 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:34.213036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:34.213552 (+ 516us) server_negotiation.cc:407] Connection header received
0807 20:10:34.214726 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:34.214727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:34.214743 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:34.214773 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:34.215929 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.216734 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.218468 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.218779 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.218828 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:34.221924 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:34.221926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:34.221926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:34.221933 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:34.223792 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:34.224289 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:34.224291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:34.224291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:34.224350 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:34.224662 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:34.224663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:34.224664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:34.224837 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:34.224883 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:34.225265 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 20:10:34.225359 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:10:34.226151 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:34.212602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34003 (local address 127.3.15.130:50903)
0807 20:10:34.213363 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:34.213422 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:34.214595 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:34.214964 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:34.214969 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:34.215391 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:34.215810 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:34.215818 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.216893 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.216894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:34.218350 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:34.218353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.218942 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.218943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:34.219099 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:34.219766 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:34.219777 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:34.221811 (+ 2034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:34.223894 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:34.223901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:34.223904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:34.224198 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:34.224436 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:34.224437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:34.224437 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:34.224583 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:34.224934 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:34.224940 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:34.225107 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 20:10:34.225734 (+ 627us) client_negotiation.cc:240] Negotiation successful
0807 20:10:34.225909 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"thread_start_us":150,"threads_started":1}
I20260807 20:10:34.226840 16461 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34003
I20260807 20:10:34.226943 16461 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:34.227139 16461 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:34.227514 16097 ts_manager.cc:194] Registered new tserver with Master: cd4467b7cfba4e0ba92fc4f926b3e5a8 (127.3.15.130:38173)
I20260807 20:10:34.228004 16097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:50903
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:10:34.258404 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34003
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:34.403664 16468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:34.404011 16468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:34.404069 16468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:34.405989 16468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:34.406078 16468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:34.406134 16468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:10:34.408322 16468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.15.148:51009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:34.408825 16468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:34.409147 16468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:34.417610 16477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:34.417753 16473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:34.417972 16474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:34.419054 16468 server_base.cc:1034] running on GCE node
I20260807 20:10:34.419363 16468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:34.419661 16468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:34.420835 16468 hybrid_clock.cc:648] HybridClock initialized: now 1786133434420796 us; error 55 us; skew 500 ppm
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:34.424683 16468 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:10:34.425305 16468 webserver.cc:460] Webserver started at http://127.3.15.131:46823/ using document root <none> and password file <none>
I20260807 20:10:34.425654 16468 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:34.425771 16468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:34.426010 16468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:34.445574 16468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "df53ea96c83848faad009fd53f64e445"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "54960e6bf44badfbd90d5ea57ab9eb39"
server_key_iv: "815b11ebefc3411abf83342bf6bd331e"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.446316 16468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "df53ea96c83848faad009fd53f64e445"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "54960e6bf44badfbd90d5ea57ab9eb39"
server_key_iv: "815b11ebefc3411abf83342bf6bd331e"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.450310 16468 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260807 20:10:34.464965 16484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.465158 16468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260807 20:10:34.465272 16468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "df53ea96c83848faad009fd53f64e445"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "54960e6bf44badfbd90d5ea57ab9eb39"
server_key_iv: "815b11ebefc3411abf83342bf6bd331e"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.465374 16468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:34.489122 16468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:34.492307 16468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:34.492547 16468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:34.492944 16468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:34.493392 16468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:34.493446 16468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.493484 16468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:34.493563 16468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.514653 16468 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:37603
I20260807 20:10:34.516599 16468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:10:34.520112 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16468
I20260807 20:10:34.520260 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:10:34.520602 3134 external_mini_cluster.cc:1428] Setting key 7ebc2441de6187d1f327748f5093c113
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:34.518512 16600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:37603 every 8 connection(s)
I20260807 20:10:34.531507 16324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:34.519312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:60979 (local address 127.3.15.190:34003)
0807 20:10:34.519361 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:34.519363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:34.519822 (+ 459us) server_negotiation.cc:407] Connection header received
0807 20:10:34.520764 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:34.520766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:34.520809 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:34.520842 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:34.521960 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.522734 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.524356 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.524651 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.524692 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:34.527940 (+ 3248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:34.527942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:34.527943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:34.527951 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:34.529888 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:34.530400 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:34.530402 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:34.530402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:34.530465 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:34.530752 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:34.530753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:34.530753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:34.530890 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:34.530943 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:34.531325 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 20:10:34.531417 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:10:34.532420 16603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:34.519286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34003 (local address 127.3.15.131:60979)
0807 20:10:34.519625 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:34.519682 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:34.520618 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:34.521067 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:34.521072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:34.521472 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:34.521830 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:34.521838 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.522856 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.522857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:34.524252 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:34.524255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.524878 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.524879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:34.525074 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:34.525833 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:34.525845 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:34.527812 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:34.530001 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:34.530008 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:34.530012 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:34.530312 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:34.530533 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:34.530534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:34.530534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:34.530680 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:34.530989 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:34.530995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:34.531166 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 20:10:34.531900 (+ 734us) client_negotiation.cc:240] Negotiation successful
0807 20:10:34.532098 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":201,"threads_started":1}
I20260807 20:10:34.533160 16601 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34003
I20260807 20:10:34.533286 16601 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:34.533495 16601 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:34.533907 16097 ts_manager.cc:194] Registered new tserver with Master: df53ea96c83848faad009fd53f64e445 (127.3.15.131:37603)
I20260807 20:10:34.534430 16097 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:60979
I20260807 20:10:34.543637 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:10:34.544402 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16183
I20260807 20:10:34.558452 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16326
I20260807 20:10:34.565416 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16468
I20260807 20:10:34.573518 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16063
I20260807 20:10:34.582792 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34003
--webserver_interface=127.3.15.190
--webserver_port=33607
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:34003
--ranger_config_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:34.668663 16604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:34.668951 16604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:34.668988 16604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:34.670140 16604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:34.670207 16604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:34.670231 16604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:34.670253 16604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:34.670282 16604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:34.671851 16604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:34003
--ranger_config_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=33607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:34.672140 16604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:34.672377 16604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:34.678396 16610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:34.678431 16604 server_base.cc:1034] running on GCE node
W20260807 20:10:34.678399 16609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:34.678553 16612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:34.678915 16604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:34.679247 16604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:34.680413 16604 hybrid_clock.cc:648] HybridClock initialized: now 1786133434680394 us; error 34 us; skew 500 ppm
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:34.683578 16604 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:34.684130 16604 webserver.cc:460] Webserver started at http://127.3.15.190:33607/ using document root <none> and password file <none>
I20260807 20:10:34.684389 16604 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:34.684463 16604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:34.734061 16604 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:34.734918 16622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.735167 16604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:34.735286 16604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "5e5560dd15cd49ee8cf624df2f7dcf61"
format_stamp: "Formatted at 2026-08-07 20:10:32 on dist-test-slave-1x4g"
server_key: "41fab11432db23e62f34aaf8ad712d9f"
server_key_iv: "e52857de9aa7bbe99fe7225564405b21"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.735718 16604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:34.751696 16604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:34.753535 16604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:34.753720 16604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:34.758083 16604 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:34003
I20260807 20:10:34.758131 16676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:34003 every 8 connection(s)
I20260807 20:10:34.758509 16604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:34.759755 16677 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:10:34.759792 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16604
I20260807 20:10:34.760100 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:43419
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:34003
--webserver_port=34717
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:34.760542 16677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Bootstrap starting.
I20260807 20:10:34.771503 16677 log.cc:824] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:34.774119 16677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:10:34.774439 16677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Bootstrap complete.
I20260807 20:10:34.777887 16677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:34.778228 16677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e5560dd15cd49ee8cf624df2f7dcf61, State: Initialized, Role: FOLLOWER
I20260807 20:10:34.778391 16677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:34.778453 16677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:10:34.778498 16677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:10:34.778545 16677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:10:34.779861 16677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:34.779973 16677 leader_election.cc:302] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e5560dd15cd49ee8cf624df2f7dcf61; no voters:
I20260807 20:10:34.780200 16677 leader_election.cc:288] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:10:34.780269 16681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:10:34.780443 16677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:34.780478 16681 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [term 2 LEADER]: Becoming Leader. State: Replica: 5e5560dd15cd49ee8cf624df2f7dcf61, State: Running, Role: LEADER
I20260807 20:10:34.780745 16681 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } }
I20260807 20:10:34.781136 16682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } } }
I20260807 20:10:34.781215 16682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:34.781714 16683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e5560dd15cd49ee8cf624df2f7dcf61. Latest consensus state: current_term: 2 leader_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e5560dd15cd49ee8cf624df2f7dcf61" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34003 } } }
I20260807 20:10:34.781790 16683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61 [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:34.781947 16691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:34.782122 16677 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 20:10:34.782481 16691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:34.782653 16691 catalog_manager.cc:1259] Loaded cluster ID: bdb323f6a0954b2d9dacaa01aa84148e
I20260807 20:10:34.782719 16691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:10:34.787416 16691 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:10:34.787781 16691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Loaded TSK: 0
I20260807 20:10:34.788156 16691 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:10:34.885365 16679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:34.885725 16679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:34.885804 16679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:34.887504 16679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:34.887625 16679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:34.887701 16679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:10:34.889912 16679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:43419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=34717
--tserver_master_addrs=127.3.15.190:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:34.890175 16679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:34.890491 16679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:34.897279 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:34.897468 16679 server_base.cc:1034] running on GCE node
W20260807 20:10:34.897636 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:34.897648 16715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:34.898010 16679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:34.898360 16679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:34.899557 16679 hybrid_clock.cc:648] HybridClock initialized: now 1786133434899525 us; error 47 us; skew 500 ppm
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:34.903014 16679 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:10:34.903564 16679 webserver.cc:460] Webserver started at http://127.3.15.129:34717/ using document root <none> and password file <none>
I20260807 20:10:34.903901 16679 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:34.903995 16679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:34.942804 16679 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260807 20:10:34.945626 16726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.945884 16679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 20:10:34.945976 16679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "37b88e7e09af400bb26628ec591741ec"
format_stamp: "Formatted at 2026-08-07 20:10:33 on dist-test-slave-1x4g"
server_key: "97ff0d4ff8695b02658150d2a29d4625"
server_key_iv: "cbbcbc6acb45007deb5409c161b8f189"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:34.946449 16679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:34.967257 16679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:34.969915 16679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:34.970146 16679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:34.970458 16679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:34.970898 16679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:34.970999 16679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.971107 16679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:34.971180 16679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:34.979208 16679 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:43419
I20260807 20:10:34.979298 16845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:43419 every 8 connection(s)
I20260807 20:10:34.979816 16679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:10:34.983227 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16679
I20260807 20:10:34.983549 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:38173
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:34003
--webserver_port=33723
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:10:34 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133434, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:34.994282 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:34.980838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34003 (local address 127.3.15.129:41829)
0807 20:10:34.981293 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:34.981537 (+ 244us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:34.982251 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:34.982833 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:34.982838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:34.983108 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:34.983426 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:34.983433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.985367 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.985368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:34.986646 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:34.986650 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.987526 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.987528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:34.987663 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:34.988318 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:34.988329 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:34.989923 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:34.992285 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:34.992292 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:34.992294 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:34.992601 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:34.992988 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:34.992989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:34.992990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:34.993126 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:34.993614 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:34.993620 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:34.993773 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 20:10:34.993861 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:10:34.994036 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":96,"threads_started":1}
I20260807 20:10:34.995211 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:34.980948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:41829 (local address 127.3.15.190:34003)
0807 20:10:34.981478 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:34.981489 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:34.981690 (+ 201us) server_negotiation.cc:407] Connection header received
0807 20:10:34.982522 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:34.982530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:34.982589 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:34.982756 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:34.983593 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.985237 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.986743 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:34.987388 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:34.989092 (+ 1704us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:34.990051 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:34.990056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:34.990057 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:34.990069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:34.992181 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:34.992717 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:34.992721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:34.992721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:34.992891 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:34.993209 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:34.993210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:34.993210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:34.993520 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:34.994767 (+ 1247us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:34.994846 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 20:10:34.994988 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":99,"threads_started":1}
I20260807 20:10:34.996155 16846 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34003
I20260807 20:10:34.996254 16846 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:34.996438 16846 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
W20260807 20:10:34.997363 16846 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:34003 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 20:10:34.997452 16846 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:34.997519 16846 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:34.997864 16846 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:34.997957 16846 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
W20260807 20:10:34.998242 16846 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:10:35.104905 16850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:35.105290 16850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:35.105379 16850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:35.107453 16850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:35.107543 16850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:35.107602 16850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:10:35.109472 16850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:38173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=33723
--tserver_master_addrs=127.3.15.190:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:35.109797 16850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:35.110090 16850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:35.117617 16856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:35.117900 16855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:35.118043 16850 server_base.cc:1034] running on GCE node
W20260807 20:10:35.117669 16858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:35.118468 16850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:35.118811 16850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:35.120036 16850 hybrid_clock.cc:648] HybridClock initialized: now 1786133435119988 us; error 45 us; skew 500 ppm
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:35.123843 16850 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:10:35.124547 16850 webserver.cc:460] Webserver started at http://127.3.15.130:33723/ using document root <none> and password file <none>
I20260807 20:10:35.124993 16850 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:35.125156 16850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:35.144861 16850 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 20:10:35.146169 16866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:35.146376 16850 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:35.146488 16850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "5c435e6e09d2a28036912463e7be53e6"
server_key_iv: "310a08bb243d20d82cf3b9b00d4261e6"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:35.146924 16850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:35.165447 16850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:35.168025 16850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:35.168256 16850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:35.168584 16850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:35.169107 16850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:35.169175 16850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:35.169224 16850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:35.169255 16850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:35.177459 16850 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:38173
I20260807 20:10:35.177659 16983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:38173 every 8 connection(s)
I20260807 20:10:35.178081 16850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:10:35.182950 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16850
I20260807 20:10:35.183324 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.15.148:51009/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:37603
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:34003
--webserver_port=46823
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:35.189641 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:35.179029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:36531 (local address 127.3.15.190:34003)
0807 20:10:35.179078 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:35.179080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:35.179409 (+ 329us) server_negotiation.cc:407] Connection header received
0807 20:10:35.180113 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:35.180115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:35.180133 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:35.180218 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:35.181057 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.181739 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.183269 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.183618 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.183693 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:35.186454 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:35.186455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:35.186456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:35.186462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:35.188153 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:35.188607 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:35.188609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:35.188609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:35.188671 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:35.188972 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:35.188973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:35.188973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:35.189096 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:35.189176 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:35.189457 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 20:10:35.189550 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:10:35.189822 16986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:35.179043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34003 (local address 127.3.15.130:36531)
0807 20:10:35.179237 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:35.179290 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:35.179971 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:35.180298 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:35.180303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:35.180632 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:35.180951 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:35.180959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.181838 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.181839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:35.183169 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:35.183172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.183738 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.183739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:35.183891 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:35.184520 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:35.184530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:35.186342 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:35.188259 (+ 1917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:35.188266 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:35.188269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:35.188505 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:35.188742 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:35.188743 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:35.188743 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:35.188898 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:35.189167 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:35.189173 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:35.189317 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 20:10:35.189422 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 20:10:35.189574 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260807 20:10:35.190603 16984 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34003
I20260807 20:10:35.190718 16984 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.190909 16984 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
W20260807 20:10:35.191596 16984 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:34003 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 20:10:35.191692 16984 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.191771 16984 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:35.192145 16984 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.192235 16984 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
W20260807 20:10:35.192725 16984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:10:35.313580 16987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:35.313947 16987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:35.314003 16987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:35.316071 16987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:35.316154 16987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:35.316205 16987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:10:35.318706 16987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.15.148:51009/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:37603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=46823
--tserver_master_addrs=127.3.15.190:34003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:35.319037 16987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:35.319444 16987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:35.325285 16994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:35.325661 16996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:35.325877 16987 server_base.cc:1034] running on GCE node
W20260807 20:10:35.326134 16993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:35.326375 16987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:35.326673 16987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:35.327857 16987 hybrid_clock.cc:648] HybridClock initialized: now 1786133435327834 us; error 31 us; skew 500 ppm
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:35.330952 16987 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:10:35.331404 16987 webserver.cc:460] Webserver started at http://127.3.15.131:46823/ using document root <none> and password file <none>
I20260807 20:10:35.331652 16987 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:35.331736 16987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:35.350447 16987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:35.351624 17004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:35.351845 16987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:10:35.351941 16987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "df53ea96c83848faad009fd53f64e445"
format_stamp: "Formatted at 2026-08-07 20:10:34 on dist-test-slave-1x4g"
server_key: "54960e6bf44badfbd90d5ea57ab9eb39"
server_key_iv: "815b11ebefc3411abf83342bf6bd331e"
server_key_version: "kuduclusterkey@0"
I20260807 20:10:35.352382 16987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:35.382962 16987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:35.385637 16987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:35.385829 16987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:35.386165 16987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:35.386612 16987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:35.386672 16987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:35.386708 16987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:35.386737 16987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:35.394945 16987 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:37603
I20260807 20:10:35.395501 16987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:10:35.400590 17124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:37603 every 8 connection(s)
I20260807 20:10:35.401592 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 16987
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:35.407547 16849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:35.396406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:49629 (local address 127.3.15.190:34003)
0807 20:10:35.396460 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:35.396462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:35.396747 (+ 285us) server_negotiation.cc:407] Connection header received
0807 20:10:35.397607 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:35.397608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:35.397622 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:35.397654 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:35.398492 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.399289 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.401057 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.401252 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.401343 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:35.404374 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:35.404376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:35.404376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:35.404384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:35.406042 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:35.406507 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:35.406508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:35.406508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:35.406573 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:35.406832 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:35.406833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:35.406833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:35.406960 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:35.407034 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:35.407364 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 20:10:35.407455 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:10:35.407747 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:35.396408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34003 (local address 127.3.15.131:49629)
0807 20:10:35.396593 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:35.396655 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:35.397455 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:35.397844 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:35.397849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:35.398089 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:35.398365 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:35.398371 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.399405 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.399406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:35.400915 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:35.400918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.401838 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.401841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:35.402061 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:35.402670 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:35.402686 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:35.404270 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:35.406164 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:35.406171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:35.406173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:35.406403 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:35.406643 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:35.406644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:35.406644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:35.406761 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:35.407037 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:35.407044 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:35.407209 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 20:10:35.407341 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 20:10:35.407500 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
I20260807 20:10:35.408406 17125 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34003
I20260807 20:10:35.408519 17125 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.408702 17125 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
W20260807 20:10:35.409472 17125 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:34003 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 20:10:35.409559 17125 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.409634 17125 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:35.409924 17125 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.409999 17125 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
W20260807 20:10:35.410305 17125 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:10:35.418430 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:35.402061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.3.15.190:34003)
0807 20:10:35.402180 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:35.402182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:35.408973 (+ 6791us) server_negotiation.cc:407] Connection header received
0807 20:10:35.409112 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:35.409113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:35.409125 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:35.409188 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:35.410167 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.410878 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.412035 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:35.412242 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:35.412299 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:35.413997 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:35.413998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:35.413998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:35.414005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:35.415787 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:35.417406 (+ 1619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:35.417407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:35.417407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:35.417481 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:35.417715 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:35.417716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:35.417716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:35.417852 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:35.418060 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:35.418113 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 20:10:35.418206 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
Aug 07 20:10:35 dist-test-slave-1x4g krb5kdc[15603](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:35.998452 16846 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:35.998644 16846 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:35.999078 16639 ts_manager.cc:194] Registered new tserver with Master: 37b88e7e09af400bb26628ec591741ec (127.3.15.129:43419)
I20260807 20:10:35.999785 16639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:41829
I20260807 20:10:36.193010 16984 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:36.193161 16984 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:36.193657 16639 ts_manager.cc:194] Registered new tserver with Master: cd4467b7cfba4e0ba92fc4f926b3e5a8 (127.3.15.130:38173)
I20260807 20:10:36.194504 16639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:36531
Aug 07 20:10:36 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:36.410501 17125 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:36.410665 17125 heartbeater.cc:511] Master 127.3.15.190:34003 requested a full tablet report, sending...
I20260807 20:10:36.411021 16639 ts_manager.cc:194] Registered new tserver with Master: df53ea96c83848faad009fd53f64e445 (127.3.15.131:37603)
I20260807 20:10:36.411584 16639 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:49629
I20260807 20:10:36.417349 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:10:36.434934 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.418348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.3.15.190:34003)
0807 20:10:36.424903 (+ 6555us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.424905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.424915 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:10:36.424938 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.424939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.424949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.425042 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:36.425746 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.426289 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.427039 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.427250 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.427326 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.431839 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:36.431841 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:36.431842 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:36.431857 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:36.433562 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:36.433935 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:36.433936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:36.433936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:36.434000 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:36.434276 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:36.434277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:36.434278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:36.434452 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:36.434567 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.434715 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.434809 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6531,"thread_start_us":74,"threads_started":1}
I20260807 20:10:36.437196 16639 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59686:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:10:36.439105 16639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:10:36.447669 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.442092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:37603 (local address 127.0.0.1:33254)
0807 20:10:36.442784 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:36.442794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:36.442846 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:36.443366 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:36.443367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:36.443369 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:36.443416 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.443418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.444702 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.444703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.446072 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.446076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.447244 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.447245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.447378 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.447399 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 20:10:36.447503 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 20:10:36.447577 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260807 20:10:36.447835 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.442090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:43419 (local address 127.0.0.1:39152)
0807 20:10:36.442309 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:36.442354 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:36.442424 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:36.443237 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:36.443241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:36.443245 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:36.443321 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.443323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.445028 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.445029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.446114 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.446116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.447635 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.447636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.447694 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.447698 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:10:36.447728 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 20:10:36.447767 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260807 20:10:36.448174 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.442505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39152 (local address 127.3.15.129:43419)
0807 20:10:36.442644 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.442650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.442661 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:10:36.443103 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.443106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.443126 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.443398 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:36.443513 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.444599 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.446206 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.447546 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.447893 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.447973 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.448002 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.448054 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260807 20:10:36.448704 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.442380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.3.15.131:37603)
0807 20:10:36.442734 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.442737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.443081 (+ 344us) server_negotiation.cc:407] Connection header received
0807 20:10:36.443204 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.443206 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.443221 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.443288 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:36.443526 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.444615 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.446173 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.447079 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.447143 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.447211 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.448546 (+ 1335us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.448594 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":89,"threads_started":1}
I20260807 20:10:36.449051 16767 tablet_service.cc:1467] Processing CreateTablet for tablet 96cf4fcaba894ca6a5464873fba65bd3 (DEFAULT_TABLE table=test-table [id=ac5303300086451692a2d7275d8e3a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:36.449533 16767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96cf4fcaba894ca6a5464873fba65bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:36.449519 17054 tablet_service.cc:1467] Processing CreateTablet for tablet 96cf4fcaba894ca6a5464873fba65bd3 (DEFAULT_TABLE table=test-table [id=ac5303300086451692a2d7275d8e3a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:36.449994 17054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96cf4fcaba894ca6a5464873fba65bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:36.453934 17158 tablet_bootstrap.cc:492] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Bootstrap starting.
I20260807 20:10:36.454106 17159 tablet_bootstrap.cc:492] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: Bootstrap starting.
I20260807 20:10:36.454972 17159 tablet_bootstrap.cc:654] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:36.455312 17158 tablet_bootstrap.cc:654] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:36.455302 17159 log.cc:824] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:36.455914 17158 log.cc:824] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:36.456851 17158 tablet_bootstrap.cc:492] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: No bootstrap required, opened a new log
I20260807 20:10:36.456948 17158 ts_tablet_manager.cc:1397] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:10:36.456894 17159 tablet_bootstrap.cc:492] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: No bootstrap required, opened a new log
I20260807 20:10:36.457134 17159 ts_tablet_manager.cc:1397] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:10:36.460629 17158 raft_consensus.cc:348] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.460834 17158 raft_consensus.cc:374] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:36.460886 17158 raft_consensus.cc:729] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df53ea96c83848faad009fd53f64e445, State: Initialized, Role: FOLLOWER
I20260807 20:10:36.460723 17159 raft_consensus.cc:348] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.461177 17159 raft_consensus.cc:374] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:36.461278 17159 raft_consensus.cc:729] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b88e7e09af400bb26628ec591741ec, State: Initialized, Role: FOLLOWER
I20260807 20:10:36.461330 17158 consensus_queue.cc:260] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.461588 17159 consensus_queue.cc:260] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.461700 17158 ts_tablet_manager.cc:1428] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 20:10:36.461829 17125 heartbeater.cc:503] Master 127.3.15.190:34003 was elected leader, sending a full tablet report...
I20260807 20:10:36.462059 17159 ts_tablet_manager.cc:1428] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 20:10:36.462217 16846 heartbeater.cc:503] Master 127.3.15.190:34003 was elected leader, sending a full tablet report...
I20260807 20:10:36.465271 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.442092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:38173 (local address 127.0.0.1:52638)
0807 20:10:36.448354 (+ 6262us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:36.448364 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:36.448401 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:36.454510 (+ 6109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:36.454511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:36.454513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:36.454555 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.454557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.456111 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.456112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.457257 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.457259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.464976 (+ 7717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.464978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.465053 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.465056 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:10:36.465131 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:10:36.465186 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6250,"mutex_wait_us":439,"thread_start_us":76,"threads_started":1}
I20260807 20:10:36.465282 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.442372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.3.15.130:38173)
0807 20:10:36.454327 (+ 11955us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.454332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.454344 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:10:36.454380 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.454382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.454398 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.454621 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:36.454814 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.455952 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.457356 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.458168 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.464975 (+ 6807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.465065 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.465148 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.465201 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11919,"thread_start_us":51,"threads_started":1}
I20260807 20:10:36.466423 16915 tablet_service.cc:1467] Processing CreateTablet for tablet 96cf4fcaba894ca6a5464873fba65bd3 (DEFAULT_TABLE table=test-table [id=ac5303300086451692a2d7275d8e3a81]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:36.466830 16915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 96cf4fcaba894ca6a5464873fba65bd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:36.470549 17164 tablet_bootstrap.cc:492] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Bootstrap starting.
I20260807 20:10:36.471966 17164 tablet_bootstrap.cc:654] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:36.472667 17164 log.cc:824] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:36.473768 17164 tablet_bootstrap.cc:492] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: No bootstrap required, opened a new log
I20260807 20:10:36.473852 17164 ts_tablet_manager.cc:1397] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:10:36.477908 17164 raft_consensus.cc:348] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.478125 17164 raft_consensus.cc:374] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:36.478186 17164 raft_consensus.cc:729] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4467b7cfba4e0ba92fc4f926b3e5a8, State: Initialized, Role: FOLLOWER
I20260807 20:10:36.478370 17164 consensus_queue.cc:260] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.478698 17164 ts_tablet_manager.cc:1428] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260807 20:10:36.478825 16984 heartbeater.cc:503] Master 127.3.15.190:34003 was elected leader, sending a full tablet report...
W20260807 20:10:36.482194 16847 tablet.cc:2369] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 20:10:36 dist-test-slave-1x4g krb5kdc[15603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133435, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for HTTP/127.3.15.148@KRBTEST.COM
I20260807 20:10:36.649979 17166 raft_consensus.cc:482] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:36.650223 17166 raft_consensus.cc:504] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.650674 17166 leader_election.cc:288] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37b88e7e09af400bb26628ec591741ec (127.3.15.129:43419), df53ea96c83848faad009fd53f64e445 (127.3.15.131:37603)
W20260807 20:10:36.651660 17126 tablet.cc:2369] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:10:36.655521 17156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.651117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:48813 (local address 127.3.15.129:43419)
0807 20:10:36.651166 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.651169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.651526 (+ 357us) server_negotiation.cc:407] Connection header received
0807 20:10:36.651659 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.651660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.651675 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.651712 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:36.652158 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.652897 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.654377 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.654964 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.655020 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.655031 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.655375 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.655424 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:10:36.656049 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.650858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:43419 (local address 127.3.15.130:48813)
0807 20:10:36.651444 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:36.651458 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:36.651562 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:36.651894 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:36.651895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:36.651897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:36.652043 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.652045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.653015 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.653016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.654254 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.654258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.655188 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.655189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.655265 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.655269 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:10:36.655890 (+ 621us) client_negotiation.cc:240] Negotiation successful
0807 20:10:36.655941 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":140,"threads_started":1}
I20260807 20:10:36.656598 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96cf4fcaba894ca6a5464873fba65bd3" candidate_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b88e7e09af400bb26628ec591741ec" is_pre_election: true
I20260807 20:10:36.656841 16792 raft_consensus.cc:2454] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4467b7cfba4e0ba92fc4f926b3e5a8 in term 0.
I20260807 20:10:36.657142 16870 leader_election.cc:302] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b88e7e09af400bb26628ec591741ec, cd4467b7cfba4e0ba92fc4f926b3e5a8; no voters:
I20260807 20:10:36.657289 17166 raft_consensus.cc:2781] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:10:36.657335 17166 raft_consensus.cc:482] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:10:36.657362 17166 raft_consensus.cc:3037] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:36.658993 17166 raft_consensus.cc:504] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.659149 17166 leader_election.cc:288] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [CANDIDATE]: Term 1 election: Requested vote from peers 37b88e7e09af400bb26628ec591741ec (127.3.15.129:43419), df53ea96c83848faad009fd53f64e445 (127.3.15.131:37603)
I20260807 20:10:36.659507 16792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96cf4fcaba894ca6a5464873fba65bd3" candidate_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37b88e7e09af400bb26628ec591741ec"
I20260807 20:10:36.659597 16792 raft_consensus.cc:3037] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:36.661058 16792 raft_consensus.cc:2454] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd4467b7cfba4e0ba92fc4f926b3e5a8 in term 1.
I20260807 20:10:36.661321 16870 leader_election.cc:302] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37b88e7e09af400bb26628ec591741ec, cd4467b7cfba4e0ba92fc4f926b3e5a8; no voters:
I20260807 20:10:36.661459 17166 raft_consensus.cc:2781] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:36.661630 17166 raft_consensus.cc:686] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 1 LEADER]: Becoming Leader. State: Replica: cd4467b7cfba4e0ba92fc4f926b3e5a8, State: Running, Role: LEADER
I20260807 20:10:36.661756 17166 consensus_queue.cc:237] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } }
I20260807 20:10:36.662757 16638 catalog_manager.cc:5495] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd4467b7cfba4e0ba92fc4f926b3e5a8 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 38173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 } health_report { overall_health: UNKNOWN } } }
I20260807 20:10:36.666646 17155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.651209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:51903 (local address 127.3.15.131:37603)
0807 20:10:36.651256 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.651258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.656534 (+ 5276us) server_negotiation.cc:407] Connection header received
0807 20:10:36.656662 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.656663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.656694 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.656728 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:36.662541 (+ 5813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.663301 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.665563 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.666230 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.666289 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.666299 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.666526 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.666571 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:10:36.667091 17169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.650904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:37603 (local address 127.3.15.130:51903)
0807 20:10:36.656476 (+ 5572us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:36.656487 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:36.656563 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:36.662365 (+ 5802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:36.662366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:36.662369 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:36.662409 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.662411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.664323 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.664324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.665441 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:36.665445 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.666369 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.666370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:36.666434 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.666438 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:10:36.666925 (+ 487us) client_negotiation.cc:240] Negotiation successful
0807 20:10:36.666972 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5559,"thread_start_us":154,"threads_started":1}
I20260807 20:10:36.667608 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96cf4fcaba894ca6a5464873fba65bd3" candidate_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df53ea96c83848faad009fd53f64e445" is_pre_election: true
I20260807 20:10:36.667789 17074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "96cf4fcaba894ca6a5464873fba65bd3" candidate_uuid: "cd4467b7cfba4e0ba92fc4f926b3e5a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df53ea96c83848faad009fd53f64e445"
I20260807 20:10:36.667845 17075 raft_consensus.cc:2454] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd4467b7cfba4e0ba92fc4f926b3e5a8 in term 0.
I20260807 20:10:36.678740 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.675099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.3.15.130:38173)
0807 20:10:36.675156 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:36.675158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:36.675166 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:10:36.675207 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:36.675207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:36.675231 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:36.675304 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:10:36.675502 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.676283 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.677611 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:36.677905 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:36.677973 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:36.678183 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:10:36.678323 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:10:36.678398 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:36.678631 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 20:10:36.678657 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 20:10:36.679672 16985 tablet.cc:2369] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:10:36.681116 17074 raft_consensus.cc:3037] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:36.682540 17074 raft_consensus.cc:1261] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 1 FOLLOWER]: Refusing update from remote peer cd4467b7cfba4e0ba92fc4f926b3e5a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:36.682865 17170 consensus_queue.cc:1035] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df53ea96c83848faad009fd53f64e445" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:36.684768 16792 raft_consensus.cc:1261] T 96cf4fcaba894ca6a5464873fba65bd3 P 37b88e7e09af400bb26628ec591741ec [term 1 FOLLOWER]: Refusing update from remote peer cd4467b7cfba4e0ba92fc4f926b3e5a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:36.685092 17170 consensus_queue.cc:1035] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "37b88e7e09af400bb26628ec591741ec" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 43419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:36.690315 17174 mvcc.cc:205] Tried to move back new op lower bound from 7316002556642684928 to 7316002556567207936. Current Snapshot: MvccSnapshot[applied={T|T < 7316002556642684928}]
I20260807 20:10:36.690662 17177 mvcc.cc:205] Tried to move back new op lower bound from 7316002556642684928 to 7316002556567207936. Current Snapshot: MvccSnapshot[applied={T|T < 7316002556642684928}]
I20260807 20:10:36.692229 17175 mvcc.cc:205] Tried to move back new op lower bound from 7316002556642684928 to 7316002556567207936. Current Snapshot: MvccSnapshot[applied={T|T < 7316002556642684928}]
I20260807 20:10:36.695467 16638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:10:36.695581 16638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59686:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:10:36.696317 16638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e5560dd15cd49ee8cf624df2f7dcf61: Sending DeleteTablet for 3 replicas of tablet 96cf4fcaba894ca6a5464873fba65bd3
I20260807 20:10:36.696910 16767 tablet_service.cc:1514] Processing DeleteTablet for tablet 96cf4fcaba894ca6a5464873fba65bd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:36 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:39152
I20260807 20:10:36.697258 17054 tablet_service.cc:1514] Processing DeleteTablet for tablet 96cf4fcaba894ca6a5464873fba65bd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:36 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:33254
I20260807 20:10:36.697623 16915 tablet_service.cc:1514] Processing DeleteTablet for tablet 96cf4fcaba894ca6a5464873fba65bd3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:36 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:52638
I20260807 20:10:36.697949 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16679
I20260807 20:10:36.713514 17186 tablet_replica.cc:330] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: stopping tablet replica
I20260807 20:10:36.713686 17186 raft_consensus.cc:2229] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:10:36.713828 17186 raft_consensus.cc:2258] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:36.714013 17184 tablet_replica.cc:330] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: stopping tablet replica
I20260807 20:10:36.714129 17184 raft_consensus.cc:2229] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:10:36.714241 17184 raft_consensus.cc:2258] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:36.714753 17186 ts_tablet_manager.cc:1905] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:36.716862 17184 ts_tablet_manager.cc:1905] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:36.718448 17184 ts_tablet_manager.cc:1918] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:36.718528 17184 log.cc:1196] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/96cf4fcaba894ca6a5464873fba65bd3
I20260807 20:10:36.718803 17184 ts_tablet_manager.cc:1939] T 96cf4fcaba894ca6a5464873fba65bd3 P df53ea96c83848faad009fd53f64e445: Deleting consensus metadata
I20260807 20:10:36.719288 16623 catalog_manager.cc:4841] TS df53ea96c83848faad009fd53f64e445 (127.3.15.131:37603): tablet 96cf4fcaba894ca6a5464873fba65bd3 (table test-table [id=ac5303300086451692a2d7275d8e3a81]) successfully deleted
I20260807 20:10:36.719489 17186 ts_tablet_manager.cc:1918] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:36.719563 17186 log.cc:1196] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/96cf4fcaba894ca6a5464873fba65bd3
I20260807 20:10:36.719826 17186 ts_tablet_manager.cc:1939] T 96cf4fcaba894ca6a5464873fba65bd3 P cd4467b7cfba4e0ba92fc4f926b3e5a8: Deleting consensus metadata
I20260807 20:10:36.720217 16624 catalog_manager.cc:4841] TS cd4467b7cfba4e0ba92fc4f926b3e5a8 (127.3.15.130:38173): tablet 96cf4fcaba894ca6a5464873fba65bd3 (table test-table [id=ac5303300086451692a2d7275d8e3a81]) successfully deleted
W20260807 20:10:36.725128 16626 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:43419 (error 108)
I20260807 20:10:36.729045 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16850
I20260807 20:10:36.743079 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16987
I20260807 20:10:36.748620 17153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:36.748472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:43419 (local address 127.0.0.1:39162)
0807 20:10:36.748520 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:36.748576 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:43419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 20:10:36.748857 16626 catalog_manager.cc:4584] TS 37b88e7e09af400bb26628ec591741ec (127.3.15.129:43419): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 96cf4fcaba894ca6a5464873fba65bd3: Network error: Client connection negotiation failed: client connection to 127.3.15.129:43419: connect: Connection refused (error 111)
I20260807 20:10:36.750380 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 16604
I20260807 20:10:36.759233 3134 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 20:10:37.104419 3134 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 20:10:37.104521 3134 mini_ranger.cc:66] Stopping Ranger...
I20260807 20:10:37.449312 3134 mini_ranger.cc:68] Stopped Ranger
2026-08-07 20:10:37.449 UTC [15634] LOG: received smart shutdown request
2026-08-07 20:10:37.454 UTC [15634] LOG: background worker "logical replication launcher" (PID 15643) exited with exit code 1
2026-08-07 20:10:37.454 UTC [15638] LOG: shutting down
2026-08-07 20:10:37.609 UTC [15634] LOG: database system is shut down
2026-08-07T20:10:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48494 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:10:37 dist-test-slave-1x4g krb5kdc[17196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:10:37 dist-test-slave-1x4g krb5kdc[17196](info): set up 2 sockets
Aug 07 20:10:37 dist-test-slave-1x4g krb5kdc[17196](info): commencing operation
krb5kdc: starting...
W20260807 20:10:38.789899 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:10:38 dist-test-slave-1x4g krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:10:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:10:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:10:38.919564 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:37143
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:37143,127.3.15.189:36747,127.3.15.188:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:39.003556 17210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:39.003796 17210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:39.003820 17210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:39.005012 17210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:39.005065 17210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:39.005080 17210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:39.005092 17210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:39.005105 17210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:39.006664 17210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:37143,127.3.15.189:36747,127.3.15.188:40007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:37143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:39.006870 17210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:39.007055 17210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:39.012176 17220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:39.012331 17216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:39.012176 17218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:39.012494 17210 server_base.cc:1034] running on GCE node
I20260807 20:10:39.012838 17210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:39.013211 17210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:39.014379 17210 hybrid_clock.cc:648] HybridClock initialized: now 1786133439014372 us; error 54 us; skew 500 ppm
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:39.017102 17210 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:39.017606 17210 webserver.cc:460] Webserver started at http://127.3.15.190:37971/ using document root <none> and password file <none>
I20260807 20:10:39.017870 17210 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:39.017940 17210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:39.018093 17210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:39.019193 17210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "645487cb2f2b4fc9b650dbef0e9b42c6"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "16013cf12ab540bea696c96f56d927e5"
server_key_iv: "183b5b303caa045a9d37739bcb8f2102"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.019578 17210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "645487cb2f2b4fc9b650dbef0e9b42c6"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "16013cf12ab540bea696c96f56d927e5"
server_key_iv: "183b5b303caa045a9d37739bcb8f2102"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.021111 17210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:10:39.022011 17227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:39.022223 17210 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:39.022338 17210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "645487cb2f2b4fc9b650dbef0e9b42c6"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "16013cf12ab540bea696c96f56d927e5"
server_key_iv: "183b5b303caa045a9d37739bcb8f2102"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.022418 17210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:39.040503 17210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:39.042636 17210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:39.042810 17210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:39.047219 17210 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:37143
I20260807 20:10:39.047257 17280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:37143 every 8 connection(s)
I20260807 20:10:39.047662 17210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:39.048717 17281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:39.050376 17281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.055428 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17210
I20260807 20:10:39.055553 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:10:39.055750 3134 external_mini_cluster.cc:1428] Setting key 3c2b16db009f6a948cbce3457cf30dcf
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:39.061319 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.051102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37143 (local address 127.0.0.1:32920)
0807 20:10:39.051304 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.051349 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.051894 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.052225 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.052228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.052520 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.052687 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.052692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.053774 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.053775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.054470 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.054472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.054971 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.054972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.055038 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.055642 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.055651 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.057525 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.059707 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.059714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.059715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.059910 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.060199 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.060200 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.060200 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.060281 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.060821 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.060828 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.060921 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.061017 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.061155 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":94,"threads_started":1}
I20260807 20:10:39.061399 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.051143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.3.15.190:37143)
0807 20:10:39.051314 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.051323 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.051476 (+ 153us) server_negotiation.cc:407] Connection header received
0807 20:10:39.052106 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.052108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.052124 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.052219 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.052807 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.053658 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.054571 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.054879 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.055002 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.057641 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.057651 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.057651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.057660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.059621 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.059995 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.059999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.060000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.060120 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.060347 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.060348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.060348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.060706 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.060823 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.061081 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.061246 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260807 20:10:39.062183 17281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.062583 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.062432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60624)
0807 20:10:39.062492 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.062550 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 20:10:39.062731 17231 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
I20260807 20:10:39.063155 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.063062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60640)
0807 20:10:39.063096 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.063126 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260807 20:10:39.063293 17281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:36747: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.15.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab.
Entry for principal kudu/127.3.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab.
I20260807 20:10:39.081961 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:36747
--webserver_interface=127.3.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab
--principal=kudu/127.3.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:37143,127.3.15.189:36747,127.3.15.188:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:39.112644 17281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } attempt: 1
I20260807 20:10:39.113263 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.113080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60652)
0807 20:10:39.113160 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.113220 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 20:10:39.113701 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.113576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60664)
0807 20:10:39.113634 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.113675 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 20:10:39.113843 17281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:36747: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
W20260807 20:10:39.168361 17288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:39.168591 17288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:39.168617 17288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:39.169873 17288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:39.169925 17288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:39.169942 17288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:39.169997 17288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:39.170030 17288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:39.171574 17288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:37143,127.3.15.189:36747,127.3.15.188:40007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab
--principal=kudu/127.3.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:36747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:39.171847 17288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:39.172106 17288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:39.177131 17294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:39.177139 17297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:39.177193 17295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:39.177304 17288 server_base.cc:1034] running on GCE node
I20260807 20:10:39.177583 17288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:39.177888 17288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:39.179036 17288 hybrid_clock.cc:648] HybridClock initialized: now 1786133439179013 us; error 35 us; skew 500 ppm
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:39.181623 17288 init.cc:377] Logged in from keytab as kudu/127.3.15.189@KRBTEST.COM (short username kudu)
I20260807 20:10:39.182075 17288 webserver.cc:460] Webserver started at http://127.3.15.189:33515/ using document root <none> and password file <none>
I20260807 20:10:39.182009 17281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } attempt: 2
I20260807 20:10:39.182401 17288 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:39.182472 17288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:39.182497 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.182325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60674)
0807 20:10:39.182413 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.182455 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 20:10:39.182646 17288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:39.182862 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.182770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60688)
0807 20:10:39.182809 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.182838 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 20:10:39.183000 17281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:36747: Network error: Client connection negotiation failed: client connection to 127.3.15.189:36747: connect: Connection refused (error 111)
I20260807 20:10:39.183851 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data/instance:
uuid: "9d745d061c4942fc86992dac13e2983a"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "5795458a1f67c9a28472d26b3a56f2f7"
server_key_iv: "7ba0192f053598395d272935d8c03ab6"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.184214 17288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal/instance:
uuid: "9d745d061c4942fc86992dac13e2983a"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "5795458a1f67c9a28472d26b3a56f2f7"
server_key_iv: "7ba0192f053598395d272935d8c03ab6"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.185643 17288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:10:39.186378 17304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:39.186558 17288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:10:39.186645 17288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal
uuid: "9d745d061c4942fc86992dac13e2983a"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "5795458a1f67c9a28472d26b3a56f2f7"
server_key_iv: "7ba0192f053598395d272935d8c03ab6"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.186697 17288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:39.205845 17288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:39.208168 17288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:39.208369 17288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:39.212986 17288 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.189:36747
I20260807 20:10:39.213021 17360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.189:36747 every 8 connection(s)
I20260807 20:10:39.213385 17288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
I20260807 20:10:39.214267 17361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:39.215863 17361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.217628 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17288
I20260807 20:10:39.217722 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-1/wal/instance
I20260807 20:10:39.218001 3134 external_mini_cluster.cc:1428] Setting key 7dbf6fa0354de388ae58f841107cd8dd
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:39.225791 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.216545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37143 (local address 127.0.0.1:32934)
0807 20:10:39.216744 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.216802 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.217238 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.217458 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.217461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.217788 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.218017 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.218022 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.218766 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.218767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.219681 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.219684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.220120 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.220121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.220279 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.220827 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.220838 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.222491 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.224235 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.224243 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.224245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.224477 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.224696 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.224697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.224697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.224879 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.225175 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.225183 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.225361 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.225483 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.225628 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260807 20:10:39.225790 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.216634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32934 (local address 127.3.15.190:37143)
0807 20:10:39.216695 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.216698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.216876 (+ 178us) server_negotiation.cc:407] Connection header received
0807 20:10:39.217319 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.217320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.217332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.217392 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.218130 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.218662 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.219798 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.220026 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.220094 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.222600 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.222602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.222603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.222611 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.224130 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.224563 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.224565 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.224565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.224627 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.224955 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.224956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.224957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.225099 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.225171 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.225561 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.225671 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:10:39.226387 17361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.15.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.188@KRBTEST.COM" created.
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:10:39.234026 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.226662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60692)
0807 20:10:39.226709 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.226719 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.226756 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.227055 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.227056 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.227173 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.227222 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.227223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.228199 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.228199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.228688 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.228690 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.229369 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.229370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.229453 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.229884 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.229890 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.231076 (+ 1186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.232837 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.232838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.232838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.233004 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.233231 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.233232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.233233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.233290 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.233682 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.233683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.233728 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.233796 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.233907 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:10:39.234131 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.226757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60692 (local address 127.3.15.189:36747)
0807 20:10:39.226891 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.226895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.226910 (+ 15us) server_negotiation.cc:407] Connection header received
0807 20:10:39.226925 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.226927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.226937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.227026 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.227289 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.228122 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.228768 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.229283 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.229371 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.231173 (+ 1802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.231182 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.231183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.231196 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.232714 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.233097 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.233101 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.233101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.233163 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.233359 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.233360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.233360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.233573 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.233693 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.233833 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.233980 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":111,"thread_start_us":87,"threads_started":1}
I20260807 20:10:39.234710 17361 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.235085 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.234973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46412)
0807 20:10:39.235014 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.235061 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260807 20:10:39.235265 17306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
I20260807 20:10:39.235746 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.235656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46418)
0807 20:10:39.235704 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.235729 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 20:10:39.235868 17361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:40007: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab.
Entry for principal kudu/127.3.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab.
I20260807 20:10:39.245769 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:40007
--webserver_interface=127.3.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab
--principal=kudu/127.3.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:37143,127.3.15.189:36747,127.3.15.188:40007
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:39.274160 17281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } attempt: 3
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:10:39.281481 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.274592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60696)
0807 20:10:39.274665 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.274678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.274760 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.274918 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.274919 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.275097 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.275168 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.275170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.275870 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.275871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.276428 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.276430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.276895 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.276897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.276983 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.277486 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.277491 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.278713 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.280483 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.280484 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.280485 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.280665 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.280895 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.280896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.280896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.280955 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.281235 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.281236 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.281279 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.281332 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.281410 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 20:10:39.281562 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.274682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.3.15.189:36747)
0807 20:10:39.274757 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.274760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.274772 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:10:39.274828 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.274829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.274842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.274908 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.275275 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.275785 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.276522 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.276763 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.276894 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.278803 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.278804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.278805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.278814 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.280375 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.280739 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.280740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.280741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.280826 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.281024 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.281025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.281025 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.281169 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.281239 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.281380 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.281469 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 20:10:39.282001 17281 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.282395 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.282294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46426)
0807 20:10:39.282344 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.282376 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260807 20:10:39.282713 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.282657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46440)
0807 20:10:39.282681 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.282701 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 20:10:39.282809 17281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:40007: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
I20260807 20:10:39.285089 17361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } attempt: 1
I20260807 20:10:39.285465 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.285356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46454)
0807 20:10:39.285412 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.285442 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 20:10:39.285805 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.285724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46466)
0807 20:10:39.285765 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.285789 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:10:39.285915 17361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:40007: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
I20260807 20:10:39.313968 17281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } attempt: 1
I20260807 20:10:39.314532 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.314369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46478)
0807 20:10:39.314437 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.314499 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 20:10:39.315052 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.314921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46488)
0807 20:10:39.315001 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.315032 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 20:10:39.315186 17281 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:40007: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
W20260807 20:10:39.333106 17368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:39.333316 17368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:39.333338 17368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:39.334417 17368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:39.334452 17368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:39.334467 17368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:39.334481 17368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:39.334492 17368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:39.335903 17368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:37143,127.3.15.189:36747,127.3.15.188:40007
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab
--principal=kudu/127.3.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:40007
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:39.336123 17368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:39.336292 17368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:39.341327 17373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:39.341475 17368 server_base.cc:1034] running on GCE node
W20260807 20:10:39.341301 17376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:39.341315 17374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:39.341830 17368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:39.342167 17368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:39.343357 17368 hybrid_clock.cc:648] HybridClock initialized: now 1786133439343349 us; error 37 us; skew 500 ppm
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:39.346248 17368 init.cc:377] Logged in from keytab as kudu/127.3.15.188@KRBTEST.COM (short username kudu)
I20260807 20:10:39.346707 17368 webserver.cc:460] Webserver started at http://127.3.15.188:37011/ using document root <none> and password file <none>
I20260807 20:10:39.346944 17368 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:39.347012 17368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:39.347160 17368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:39.348254 17368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data/instance:
uuid: "b5b89d67e21044d7b59b6457f706e46d"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "5b81d0e5ee73f3d73c31a98e45408e72"
server_key_iv: "ac322318e6b802d3f2aaf8ee012475cb"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.348637 17368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal/instance:
uuid: "b5b89d67e21044d7b59b6457f706e46d"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "5b81d0e5ee73f3d73c31a98e45408e72"
server_key_iv: "ac322318e6b802d3f2aaf8ee012475cb"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.350133 17368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:39.350847 17383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:39.351060 17368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:10:39.351166 17368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal
uuid: "b5b89d67e21044d7b59b6457f706e46d"
format_stamp: "Formatted at 2026-08-07 20:10:39 on dist-test-slave-1x4g"
server_key: "5b81d0e5ee73f3d73c31a98e45408e72"
server_key_iv: "ac322318e6b802d3f2aaf8ee012475cb"
server_key_version: "encryptionkey@0"
I20260807 20:10:39.351239 17368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:39.354063 17361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } attempt: 2
I20260807 20:10:39.354440 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.354315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46492)
0807 20:10:39.354375 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.354413 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:10:39.354825 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.354721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46500)
0807 20:10:39.354769 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.354802 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:10:39.354921 17361 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:40007: Network error: Client connection negotiation failed: client connection to 127.3.15.188:40007: connect: Connection refused (error 111)
I20260807 20:10:39.370083 17368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:39.372500 17368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:39.372663 17368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:39.376888 17368 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.188:40007
I20260807 20:10:39.376945 17436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.188:40007 every 8 connection(s)
I20260807 20:10:39.377339 17368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
I20260807 20:10:39.378240 17437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:39.379980 17437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.381976 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17368
I20260807 20:10:39.382128 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786133281907902-3134-0/minicluster-data/master-2/wal/instance
I20260807 20:10:39.382306 3134 external_mini_cluster.cc:1428] Setting key 71abfacfc459d9fd161b83a46f6aa458
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:10:39.390419 17281 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } attempt: 2
I20260807 20:10:39.391193 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.380757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37143 (local address 127.0.0.1:32938)
0807 20:10:39.380957 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.381001 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.381488 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.381745 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.381748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.382021 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.382318 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.382323 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.383281 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.383282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.384391 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.384394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.384823 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.384824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.384967 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.385497 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.385505 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.387313 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.389289 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.389297 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.389301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.389661 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.389933 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.389933 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.389934 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.390064 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.390469 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.390476 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.390642 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.390750 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.390914 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
I20260807 20:10:39.391530 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.380827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32938 (local address 127.3.15.190:37143)
0807 20:10:39.380905 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.380907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.381166 (+ 259us) server_negotiation.cc:407] Connection header received
0807 20:10:39.381601 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.381602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.381613 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.381678 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.382460 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.383206 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.384493 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.384704 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.384767 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.387409 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.387411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.387411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.387420 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.389169 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.389780 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.389782 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.389782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.389851 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.390197 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.390199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.390199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.390343 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.390436 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.391358 (+ 922us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.391442 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 20:10:39.392165 17437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.392458 17441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.382788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.3.15.188:40007)
0807 20:10:39.382966 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.382971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.383002 (+ 31us) server_negotiation.cc:407] Connection header received
0807 20:10:39.383086 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.383088 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.383108 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.383173 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.383769 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.384732 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.385712 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.386172 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.386246 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.388628 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.388636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.388637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.388644 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.390662 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.391156 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.391159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.391160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.391227 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.391669 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.391670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.391671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.391863 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.392016 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.392257 (+ 241us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.392346 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":117,"threads_started":1}
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:39.399914 17283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.390746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:46526)
0807 20:10:39.390792 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.390804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.390841 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.391208 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.391209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.391382 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.391435 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.391437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.392383 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.392385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.393291 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.393313 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.393794 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.393795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.393853 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.394312 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.394317 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.395738 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.398642 (+ 2904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.398643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.398644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.398840 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.399085 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.399086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.399086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.399161 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.399582 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.399583 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.399633 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.399702 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.399799 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 20:10:39.400296 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.392403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:36747 (local address 127.0.0.1:60712)
0807 20:10:39.392492 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.392502 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.392575 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.392706 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.392707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.392964 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.393018 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.393020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.393916 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.393917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.394708 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.394711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.395235 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.395237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.395313 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.395756 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.395761 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.397184 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.399132 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.399133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.399134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.399347 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.399585 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.399586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.399586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.399647 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.399926 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.399927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.399984 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.400096 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.400196 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75,"mutex_wait_us":25}
I20260807 20:10:39.400310 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.392527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60712 (local address 127.3.15.189:36747)
0807 20:10:39.392577 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.392579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.392601 (+ 22us) server_negotiation.cc:407] Connection header received
0807 20:10:39.392631 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.392632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.392642 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.392698 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.393125 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.393831 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.394805 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.395145 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.395219 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.397297 (+ 2078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.397298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.397299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.397307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.399032 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.399435 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.399437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.399437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.399505 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.399721 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.399722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.399722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.399850 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.399934 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.400132 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.400215 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:10:39.400563 17442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.390856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46526 (local address 127.3.15.188:40007)
0807 20:10:39.391021 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.391023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.391030 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:10:39.391081 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.391082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.391092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.391193 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.391529 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.392274 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.393401 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.393706 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.393997 (+ 291us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.395831 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.395832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.395832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.395840 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.398548 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.398938 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.398939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.398939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.399009 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.399234 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.399235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.399235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.399362 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.399518 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.400326 (+ 808us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.400418 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":146,"thread_start_us":95,"threads_started":1}
I20260807 20:10:39.401057 17437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:10:39.402316 17281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: Bootstrap starting.
I20260807 20:10:39.403110 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.393616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.3.15.190:37143)
0807 20:10:39.393651 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.393653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.393744 (+ 91us) server_negotiation.cc:407] Connection header received
0807 20:10:39.393805 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.393806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.393816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.393870 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.394544 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.395232 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.396296 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.396538 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.396607 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.400017 (+ 3410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.400018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.400019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.400026 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.401669 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.401973 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.401974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.401974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.402033 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.402252 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.402253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.402253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.402360 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.402954 (+ 594us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.402989 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.403047 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:10:39.403164 17281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:39.403538 17281 log.cc:824] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:39.404577 17281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: No bootstrap required, opened a new log
I20260807 20:10:39.406910 17281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.407076 17281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:39.407152 17281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 645487cb2f2b4fc9b650dbef0e9b42c6, State: Initialized, Role: FOLLOWER
I20260807 20:10:39.407335 17281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.407675 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.407750 17281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:39.407859 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
W20260807 20:10:39.410517 17457 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:10:39.410615 17457 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:10:39.411588 17439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.401325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:33818)
0807 20:10:39.401374 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.401384 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.401426 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.404164 (+ 2738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.404165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.404937 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.404969 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.404971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.405581 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.405582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.406037 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.406039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.406429 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.406429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.406477 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.406892 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.406896 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.408263 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.410092 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.410092 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.410093 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.410319 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.410642 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.410642 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.410643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.410711 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.411308 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.411310 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.411345 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.411403 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.411491 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260807 20:10:39.412160 17442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.403861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33818 (local address 127.3.15.188:40007)
0807 20:10:39.404056 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.404057 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.404063 (+ 6us) server_negotiation.cc:407] Connection header received
0807 20:10:39.404079 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.404079 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.404086 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.404161 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.405049 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.405499 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.406137 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.406358 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.406432 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.408362 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.408364 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.408364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.408370 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.409998 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.410405 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.410406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.410407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.410488 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.410778 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.410779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.410779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.411212 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.411282 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.411981 (+ 699us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.412069 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260807 20:10:39.413863 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.403856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.3.15.189:36747)
0807 20:10:39.403902 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.403904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.403912 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:10:39.403934 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.403935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.403945 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.404014 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.404530 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.405314 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.406369 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.406687 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.406735 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.409793 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.409795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.409796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.409804 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.411836 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.412278 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.412280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.412280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.412346 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.412569 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.412570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.412570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.412703 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.413405 (+ 702us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.413676 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.413765 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:10:39.411613 17210 master_runner.cc:186] Error getting master registration for 127.3.15.189:36747: OK, instance_id { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" instance_seqno: 1786133439211809 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:10:39.414582 17210 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:10:39.415716 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:10:39.416250 17437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: Bootstrap starting.
I20260807 20:10:39.417110 17437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:39.417421 17437 log.cc:824] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:39.418324 17437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: No bootstrap required, opened a new log
I20260807 20:10:39.420368 17437 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.420517 17437 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:39.420552 17437 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5b89d67e21044d7b59b6457f706e46d, State: Initialized, Role: FOLLOWER
I20260807 20:10:39.420821 17437 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.421365 17437 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: configured and running, proceeding with master startup.
W20260807 20:10:39.423671 17478 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:10:39.423774 17478 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:10:39.424427 17467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.424577 17467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:10:39.425172 17368 master_runner.cc:186] Error getting master registration for 127.3.15.189:36747: OK, instance_id { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" instance_seqno: 1786133439211809 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:10:39.429960 17368 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:10:39.430056 17442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.417710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.3.15.188:40007)
0807 20:10:39.417745 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.417748 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.418176 (+ 428us) server_negotiation.cc:407] Connection header received
0807 20:10:39.418255 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.418256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.418268 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.418324 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.419258 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.420094 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.421241 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.421554 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.421599 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.423812 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.423814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.423814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.423822 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.426343 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.427695 (+ 1352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.427696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.427696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.427759 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.428379 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.428380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.428380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.428540 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.428660 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.429799 (+ 1139us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.429889 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:10:39.430758 17365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.417920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57822 (local address 127.3.15.189:36747)
0807 20:10:39.418046 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.418048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.419126 (+ 1078us) server_negotiation.cc:407] Connection header received
0807 20:10:39.419238 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.419239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.419251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.419284 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.422160 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.422927 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.425510 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.425835 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.425888 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.427710 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.427712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.427712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.427719 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.429421 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.429849 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.429850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.429850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.429922 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.430127 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.430128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.430128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.430278 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.430346 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.430564 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.430642 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:10:39.431707 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.417336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39078 (local address 127.3.15.190:37143)
0807 20:10:39.417398 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.417401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.418078 (+ 677us) server_negotiation.cc:407] Connection header received
0807 20:10:39.418345 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.418346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.418359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.418394 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.420215 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.421052 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.422113 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.422411 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.422478 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.427433 (+ 4955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.427435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.427435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.427443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.429429 (+ 1986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.430776 (+ 1347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.430777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.430777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.430838 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.431069 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.431070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.431070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.431204 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.431271 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.431511 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.431597 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:10:39.446082 17361 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } attempt: 3
Aug 07 20:10:39 dist-test-slave-1x4g krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133439, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:10:39.453375 17363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.446451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:40007 (local address 127.0.0.1:33838)
0807 20:10:39.446508 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:39.446521 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:39.446567 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:39.446759 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:39.446760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:39.447017 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:39.447076 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.447078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.447836 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.447837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.448336 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:39.448338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.448717 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.448718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:39.448796 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.449274 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:39.449282 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:39.450634 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:39.452266 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.452267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.452268 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.452446 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.452668 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:39.452669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:39.452669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:39.452735 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:39.453043 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:39.453045 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:39.453092 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 20:10:39.453177 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 20:10:39.453278 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:10:39.453375 17442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:39.446490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.3.15.188:40007)
0807 20:10:39.446545 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:39.446547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:39.446557 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:10:39.446645 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:39.446646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:39.446657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:39.446706 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:39.447172 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.447741 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.448463 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:39.448641 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:39.448719 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:39.450727 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:39.450729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:39.450729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:39.450736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:39.452162 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.452532 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.452533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.452533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.452594 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:39.452844 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:39.452845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:39.452846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:39.452962 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:39.453041 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:39.453204 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 20:10:39.453278 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:10:39.455401 17361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a: Bootstrap starting.
I20260807 20:10:39.456269 17361 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:39.456624 17361 log.cc:824] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:39.457485 17361 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a: No bootstrap required, opened a new log
I20260807 20:10:39.458914 17361 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.459052 17361 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:39.459200 17361 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d745d061c4942fc86992dac13e2983a, State: Initialized, Role: FOLLOWER
I20260807 20:10:39.459355 17361 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.459659 17481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.459736 17361 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:39.459800 17481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:10:39.462739 17492 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:10:39.462834 17492 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:10:39.463843 17288 master_runner.cc:263] No leader master found from master 9d745d061c4942fc86992dac13e2983a
I20260807 20:10:39.463894 17288 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:10:39.469383 17481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:39.469498 17481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.469732 17481 leader_election.cc:288] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 645487cb2f2b4fc9b650dbef0e9b42c6 (127.3.15.190:37143), b5b89d67e21044d7b59b6457f706e46d (127.3.15.188:40007)
I20260807 20:10:39.470166 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d745d061c4942fc86992dac13e2983a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" is_pre_election: true
I20260807 20:10:39.470337 17255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d745d061c4942fc86992dac13e2983a in term 0.
I20260807 20:10:39.470672 17305 leader_election.cc:302] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645487cb2f2b4fc9b650dbef0e9b42c6, 9d745d061c4942fc86992dac13e2983a; no voters:
I20260807 20:10:39.471468 17411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d745d061c4942fc86992dac13e2983a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b89d67e21044d7b59b6457f706e46d" is_pre_election: true
I20260807 20:10:39.471623 17411 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d745d061c4942fc86992dac13e2983a in term 0.
I20260807 20:10:39.472002 17481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:10:39.472070 17481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:10:39.472098 17481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:39.473066 17481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.473181 17481 leader_election.cc:288] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [CANDIDATE]: Term 1 election: Requested vote from peers 645487cb2f2b4fc9b650dbef0e9b42c6 (127.3.15.190:37143), b5b89d67e21044d7b59b6457f706e46d (127.3.15.188:40007)
I20260807 20:10:39.473508 17255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d745d061c4942fc86992dac13e2983a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6"
I20260807 20:10:39.473599 17255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:39.474509 17255 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d745d061c4942fc86992dac13e2983a in term 1.
I20260807 20:10:39.474696 17305 leader_election.cc:302] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 645487cb2f2b4fc9b650dbef0e9b42c6, 9d745d061c4942fc86992dac13e2983a; no voters:
I20260807 20:10:39.474881 17411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9d745d061c4942fc86992dac13e2983a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5b89d67e21044d7b59b6457f706e46d"
I20260807 20:10:39.474957 17411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:39.475870 17411 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d745d061c4942fc86992dac13e2983a in term 1.
I20260807 20:10:39.476130 17481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:39.476297 17481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [term 1 LEADER]: Becoming Leader. State: Replica: 9d745d061c4942fc86992dac13e2983a, State: Running, Role: LEADER
I20260807 20:10:39.476383 17481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } }
I20260807 20:10:39.476845 17481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d745d061c4942fc86992dac13e2983a. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.476917 17481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:39.477222 17496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:39.477468 17496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:39.479401 17411 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [term 1 FOLLOWER]: Refusing update from remote peer 9d745d061c4942fc86992dac13e2983a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:39.479770 17481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:39.481098 17467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d745d061c4942fc86992dac13e2983a. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.481184 17467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:10:39.481700 17255 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [term 1 FOLLOWER]: Refusing update from remote peer 9d745d061c4942fc86992dac13e2983a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:39.482031 17481 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [LEADER]: Connected to new peer: Peer: permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:39.483250 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9d745d061c4942fc86992dac13e2983a. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.483348 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:10:39.486250 17481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.486331 17481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:39.486366 17481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.486418 17481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:39.486927 17444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.487016 17444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:10:39.487429 17467 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9d745d061c4942fc86992dac13e2983a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "645487cb2f2b4fc9b650dbef0e9b42c6" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37143 } } peers { permanent_uuid: "9d745d061c4942fc86992dac13e2983a" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 36747 } } peers { permanent_uuid: "b5b89d67e21044d7b59b6457f706e46d" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 40007 } } }
I20260807 20:10:39.487507 17467 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:10:39.487811 17497 mvcc.cc:205] Tried to move back new op lower bound from 7316002568105402368 to 7316002568095883264. Current Snapshot: MvccSnapshot[applied={T|T < 7316002568105402368}]
I20260807 20:10:39.488014 17500 mvcc.cc:205] Tried to move back new op lower bound from 7316002568105402368 to 7316002568095883264. Current Snapshot: MvccSnapshot[applied={T|T < 7316002568105402368}]
I20260807 20:10:39.488561 17498 mvcc.cc:205] Tried to move back new op lower bound from 7316002568105402368 to 7316002568095883264. Current Snapshot: MvccSnapshot[applied={T|T < 7316002568105402368}]
I20260807 20:10:39.489555 17496 catalog_manager.cc:1347] Generated new cluster ID: 648b93a31f74481fae65b6ecf412f08f
I20260807 20:10:39.489602 17496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:10:39.526714 17496 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:10:39.527582 17496 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:10:39.533890 17496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9d745d061c4942fc86992dac13e2983a: Generated new TSK 0
I20260807 20:10:39.534183 17496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:10:40.411098 17457 catalog_manager.cc:1259] Loaded cluster ID: 648b93a31f74481fae65b6ecf412f08f
I20260807 20:10:40.411173 17457 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: loading cluster ID for follower catalog manager: success
I20260807 20:10:40.412017 17457 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: acquiring CA information for follower catalog manager: success
I20260807 20:10:40.412366 17457 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 645487cb2f2b4fc9b650dbef0e9b42c6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:10:40.424304 17478 catalog_manager.cc:1259] Loaded cluster ID: 648b93a31f74481fae65b6ecf412f08f
I20260807 20:10:40.424372 17478 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: loading cluster ID for follower catalog manager: success
I20260807 20:10:40.425395 17478 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: acquiring CA information for follower catalog manager: success
I20260807 20:10:40.425729 17478 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b5b89d67e21044d7b59b6457f706e46d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:10:40.588168 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17210
I20260807 20:10:40.595849 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17288
I20260807 20:10:40.601619 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17368
2026-08-07T20:10:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2881 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:10:40 dist-test-slave-1x4g krb5kdc[17513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:10:40 dist-test-slave-1x4g krb5kdc[17513](info): set up 2 sockets
Aug 07 20:10:40 dist-test-slave-1x4g krb5kdc[17513](info): commencing operation
krb5kdc: starting...
W20260807 20:10:41.682406 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:10:41 dist-test-slave-1x4g krb5kdc[17513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:10:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:10:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:10:41.811646 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42945
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:43887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:42945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:41.895565 17526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:41.895835 17526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:41.895874 17526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:41.897084 17526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:41.897161 17526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:41.897183 17526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:41.897205 17526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:41.897235 17526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:41.898759 17526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:43887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:42945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:41.899056 17526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:41.899304 17526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:41.904554 17532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:41.904551 17535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:41.904558 17533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:41.904836 17526 server_base.cc:1034] running on GCE node
I20260807 20:10:41.905164 17526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:41.905534 17526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:41.906708 17526 hybrid_clock.cc:648] HybridClock initialized: now 1786133441906678 us; error 40 us; skew 500 ppm
Aug 07 20:10:41 dist-test-slave-1x4g krb5kdc[17513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133441, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:41.913364 17526 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:41.913828 17526 webserver.cc:460] Webserver started at http://127.3.15.190:38999/ using document root <none> and password file <none>
I20260807 20:10:41.914098 17526 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:41.914147 17526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:41.914287 17526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:41.915516 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "7e9d2b534321475cba046901a124b24c"
format_stamp: "Formatted at 2026-08-07 20:10:41 on dist-test-slave-1x4g"
server_key: "5e089db4e0bb323554392a37fe753466"
server_key_iv: "24b1d0b6b841ff66826e55590676bf96"
server_key_version: "encryptionkey@0"
I20260807 20:10:41.915907 17526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "7e9d2b534321475cba046901a124b24c"
format_stamp: "Formatted at 2026-08-07 20:10:41 on dist-test-slave-1x4g"
server_key: "5e089db4e0bb323554392a37fe753466"
server_key_iv: "24b1d0b6b841ff66826e55590676bf96"
server_key_version: "encryptionkey@0"
I20260807 20:10:41.917459 17526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:10:41.918336 17543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:41.918538 17526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:41.918649 17526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "7e9d2b534321475cba046901a124b24c"
format_stamp: "Formatted at 2026-08-07 20:10:41 on dist-test-slave-1x4g"
server_key: "5e089db4e0bb323554392a37fe753466"
server_key_iv: "24b1d0b6b841ff66826e55590676bf96"
server_key_version: "encryptionkey@0"
I20260807 20:10:41.918735 17526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:41.949597 17526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:41.951655 17526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:41.951829 17526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:41.956562 17526 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:42945
I20260807 20:10:41.956637 17597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:42945 every 8 connection(s)
I20260807 20:10:41.957080 17526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:41.957563 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17526
I20260807 20:10:41.957651 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:10:41.957908 3134 external_mini_cluster.cc:1428] Setting key 7422b79eca91181f7e13001dd45f1e4c
I20260807 20:10:41.958217 17598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:41.961639 17598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c: Bootstrap starting.
I20260807 20:10:41.962353 17598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:41.962697 17598 log.cc:824] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:41.963469 17598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c: No bootstrap required, opened a new log
Aug 07 20:10:41 dist-test-slave-1x4g krb5kdc[17513](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:41.966326 17598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9d2b534321475cba046901a124b24c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42945 } }
I20260807 20:10:41.966523 17598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:41.966567 17598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e9d2b534321475cba046901a124b24c, State: Initialized, Role: FOLLOWER
I20260807 20:10:41.966727 17598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9d2b534321475cba046901a124b24c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42945 } }
I20260807 20:10:41.966802 17598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:10:41.966854 17598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:10:41.966908 17598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:41.967710 17598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9d2b534321475cba046901a124b24c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42945 } }
I20260807 20:10:41.967823 17598 leader_election.cc:302] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e9d2b534321475cba046901a124b24c; no voters:
I20260807 20:10:41.967993 17598 leader_election.cc:288] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:10:41.968091 17604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:41.968271 17598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:41.968321 17604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [term 1 LEADER]: Becoming Leader. State: Replica: 7e9d2b534321475cba046901a124b24c, State: Running, Role: LEADER
I20260807 20:10:41.968524 17604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9d2b534321475cba046901a124b24c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42945 } }
I20260807 20:10:41.968926 17605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e9d2b534321475cba046901a124b24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9d2b534321475cba046901a124b24c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42945 } } }
I20260807 20:10:41.969035 17605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:41.969304 17606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e9d2b534321475cba046901a124b24c. Latest consensus state: current_term: 1 leader_uuid: "7e9d2b534321475cba046901a124b24c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e9d2b534321475cba046901a124b24c" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42945 } } }
I20260807 20:10:41.969679 17606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c [sys.catalog]: This master's current role is: LEADER
W20260807 20:10:41.970485 17620 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:10:41.970583 17620 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:10:41.970738 17601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:41.958552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.3.15.190:42945)
0807 20:10:41.958754 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:41.958764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:41.958791 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:10:41.959403 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:41.959412 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:41.959466 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:41.959588 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:41.960466 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:41.961535 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:41.962625 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:41.963118 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:41.963203 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:41.966379 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:41.966383 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:41.966384 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:41.966396 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:41.968562 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:41.969064 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:41.969067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:41.969067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:41.969186 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:41.969456 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:41.969457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:41.969457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:41.969808 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:41.969969 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:41.970440 (+ 471us) server_negotiation.cc:299] Negotiation successful
0807 20:10:41.970556 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260807 20:10:41.970717 17613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:41.970927 17613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:41.971786 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:10:41.972640 17613 catalog_manager.cc:1347] Generated new cluster ID: 6cc213cbb0284e368e16e79b45d3c4de
I20260807 20:10:41.972695 17613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:10:42.012259 17613 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:10:42.012907 17613 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:10:42.017568 17613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e9d2b534321475cba046901a124b24c: Generated new TSK 0
I20260807 20:10:42.017765 17613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:10:42.025966 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17526
2026-08-07T20:10:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1423 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:10:42 dist-test-slave-1x4g krb5kdc[17626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:10:42 dist-test-slave-1x4g krb5kdc[17626](info): set up 2 sockets
Aug 07 20:10:42 dist-test-slave-1x4g krb5kdc[17626](info): commencing operation
krb5kdc: starting...
W20260807 20:10:43.102418 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:10:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:10:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:10:43.232048 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:46207
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:46207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:43.317291 17640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:43.317538 17640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:43.317577 17640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:43.318706 17640 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:43.318770 17640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:43.318797 17640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:43.318830 17640 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:43.318856 17640 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:43.320299 17640 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:46207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:46207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:43.320575 17640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:43.320861 17640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:43.325762 17645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:43.325872 17640 server_base.cc:1034] running on GCE node
W20260807 20:10:43.325812 17646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:43.325954 17648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:43.326177 17640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:43.326515 17640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:43.327653 17640 hybrid_clock.cc:648] HybridClock initialized: now 1786133443327641 us; error 28 us; skew 500 ppm
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:43.330296 17640 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:43.330734 17640 webserver.cc:460] Webserver started at http://127.3.15.190:36353/ using document root <none> and password file <none>
I20260807 20:10:43.330961 17640 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:43.331029 17640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:43.331233 17640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:43.332288 17640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "e593bd2e00a14db0b624be17e624d48d"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "86203971a2c5c58d15c69a732c82aacb"
server_key_iv: "d929835e16643c5eb2a6ad8d6305eb3a"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.332670 17640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "e593bd2e00a14db0b624be17e624d48d"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "86203971a2c5c58d15c69a732c82aacb"
server_key_iv: "d929835e16643c5eb2a6ad8d6305eb3a"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.334105 17640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:43.334856 17655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.335032 17640 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:10:43.335132 17640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "e593bd2e00a14db0b624be17e624d48d"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "86203971a2c5c58d15c69a732c82aacb"
server_key_iv: "d929835e16643c5eb2a6ad8d6305eb3a"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.335215 17640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:43.352174 17640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:43.354581 17640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:43.354750 17640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:43.359125 17640 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:46207
I20260807 20:10:43.359161 17709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:46207 every 8 connection(s)
I20260807 20:10:43.359537 17640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:43.360584 17710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:43.362954 17710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap starting.
I20260807 20:10:43.363488 17710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:43.363730 17710 log.cc:824] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:43.364399 17710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: No bootstrap required, opened a new log
I20260807 20:10:43.366026 17710 raft_consensus.cc:348] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:43.366169 17710 raft_consensus.cc:374] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:43.366192 17710 raft_consensus.cc:729] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e593bd2e00a14db0b624be17e624d48d, State: Initialized, Role: FOLLOWER
I20260807 20:10:43.366331 17710 consensus_queue.cc:260] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:43.366412 17710 raft_consensus.cc:388] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:10:43.366436 17710 raft_consensus.cc:482] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:10:43.366465 17710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:43.367249 17710 raft_consensus.cc:504] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:43.367341 17710 leader_election.cc:302] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e593bd2e00a14db0b624be17e624d48d; no voters:
I20260807 20:10:43.367462 17710 leader_election.cc:288] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:10:43.367537 17713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:43.367697 17713 raft_consensus.cc:686] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 LEADER]: Becoming Leader. State: Replica: e593bd2e00a14db0b624be17e624d48d, State: Running, Role: LEADER
I20260807 20:10:43.367777 17710 sys_catalog.cc:564] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:43.367836 17713 consensus_queue.cc:237] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:43.368227 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17640
I20260807 20:10:43.368367 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:10:43.368608 3134 external_mini_cluster.cc:1428] Setting key ac0a135b88efefa73fecb05906a880e1
I20260807 20:10:43.368564 17715 sys_catalog.cc:455] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e593bd2e00a14db0b624be17e624d48d. Latest consensus state: current_term: 1 leader_uuid: "e593bd2e00a14db0b624be17e624d48d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } } }
I20260807 20:10:43.368645 17715 sys_catalog.cc:458] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:43.369047 17714 sys_catalog.cc:455] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e593bd2e00a14db0b624be17e624d48d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } } }
I20260807 20:10:43.369129 17714 sys_catalog.cc:458] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:43.369277 17723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:43.369964 17723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:43.371595 17723 catalog_manager.cc:1347] Generated new cluster ID: 32071f15414a4e8ebddfc26fa7fa8cfd
I20260807 20:10:43.371649 17723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:43.378988 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.369344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.3.15.190:46207)
0807 20:10:43.369513 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.369524 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.369625 (+ 101us) server_negotiation.cc:407] Connection header received
0807 20:10:43.369785 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.369790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.369834 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.369977 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:43.370621 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.371829 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.372582 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.373090 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.373224 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.375573 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:43.375577 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:43.375577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:43.375601 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:43.377358 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.377740 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.377744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.377744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.377853 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.378081 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.378082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.378083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.378339 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:43.378508 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.378632 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.378783 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260807 20:10:43.391239 17723 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:10:43.391800 17723 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:10:43.399502 17723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Generated new TSK 0
I20260807 20:10:43.399720 17723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:10:43.430940 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:46207
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:43.515484 17737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:43.515740 17737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:43.515780 17737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:43.516988 17737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:43.517057 17737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:43.517102 17737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:10:43.518574 17737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:43.518848 17737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:43.519099 17737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:43.524502 17742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:43.524500 17743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:43.524560 17737 server_base.cc:1034] running on GCE node
W20260807 20:10:43.524478 17745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:43.524914 17737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:43.525239 17737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:43.526439 17737 hybrid_clock.cc:648] HybridClock initialized: now 1786133443526426 us; error 40 us; skew 500 ppm
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:43.529234 17737 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:10:43.529702 17737 webserver.cc:460] Webserver started at http://127.3.15.129:40453/ using document root <none> and password file <none>
I20260807 20:10:43.529943 17737 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:43.530021 17737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:43.530176 17737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:43.531280 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "8184af9eee254c2b99a8f04d39d56a70"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "7f0c1f351d415401edfd78406d2e2fc0"
server_key_iv: "9a5bad36fb075568f43a6d878455571a"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.531661 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "8184af9eee254c2b99a8f04d39d56a70"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "7f0c1f351d415401edfd78406d2e2fc0"
server_key_iv: "9a5bad36fb075568f43a6d878455571a"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.533075 17737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:10:43.533828 17752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.533998 17737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:10:43.534106 17737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "8184af9eee254c2b99a8f04d39d56a70"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "7f0c1f351d415401edfd78406d2e2fc0"
server_key_iv: "9a5bad36fb075568f43a6d878455571a"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.534179 17737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:43.540669 17737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:43.542469 17737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:43.542631 17737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:43.542848 17737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:43.543159 17737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:43.543215 17737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.543249 17737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:43.543277 17737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.549489 17737 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40817
I20260807 20:10:43.549540 17872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40817 every 8 connection(s)
I20260807 20:10:43.549873 17737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:43.556993 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17737
I20260807 20:10:43.557116 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:10:43.557330 3134 external_mini_cluster.cc:1428] Setting key 5526351f376b7e2bc7d7526a470405ea
I20260807 20:10:43.560142 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.550603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:59305 (local address 127.3.15.190:46207)
0807 20:10:43.550693 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.550696 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.550883 (+ 187us) server_negotiation.cc:407] Connection header received
0807 20:10:43.551448 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.551450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.551463 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.551510 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:43.552534 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.553133 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.554166 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.554383 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.554469 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.556726 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:43.556728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:43.556728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:43.556736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:43.558687 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.559090 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.559091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.559092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.559161 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.559401 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.559402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.559403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.559560 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:43.559627 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.559898 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.560008 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 20:10:43.560377 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.550564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:59305)
0807 20:10:43.550756 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:43.550799 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:43.551325 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:43.551616 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:43.551619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:43.552223 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:43.552421 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.552427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.553228 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.553230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.554052 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.554054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.554491 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.554492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.554623 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.555179 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:43.555188 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:43.556625 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:43.558767 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:43.558774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:43.558776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:43.559012 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:43.559229 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:43.559230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:43.559230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:43.559337 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:43.559629 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:43.559635 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:43.559764 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 20:10:43.559852 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:10:43.559981 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260807 20:10:43.560876 17873 heartbeater.cc:347] Connected to a master server at 127.3.15.190:46207
I20260807 20:10:43.560999 17873 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:43.561193 17873 heartbeater.cc:511] Master 127.3.15.190:46207 requested a full tablet report, sending...
I20260807 20:10:43.561801 17673 ts_manager.cc:194] Registered new tserver with Master: 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817)
I20260807 20:10:43.562438 17673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:59305
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:10:43.585927 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:46207
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:43.669682 17879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:43.669898 17879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:43.669921 17879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:43.671105 17879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:43.671150 17879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:43.671176 17879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:10:43.672650 17879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:43.672932 17879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:43.673182 17879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:43.678387 17885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:43.678387 17887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:43.678407 17884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:43.678705 17879 server_base.cc:1034] running on GCE node
I20260807 20:10:43.678983 17879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:43.679224 17879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:43.680389 17879 hybrid_clock.cc:648] HybridClock initialized: now 1786133443680363 us; error 59 us; skew 500 ppm
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:43.683027 17879 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:10:43.683521 17879 webserver.cc:460] Webserver started at http://127.3.15.130:35341/ using document root <none> and password file <none>
I20260807 20:10:43.683774 17879 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:43.683843 17879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:43.683990 17879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:43.685133 17879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "4f3235b0d1154209aeb79ee7a561ff81"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "85ebf1abf284c97fefd0dba861324fdd"
server_key_iv: "a983b0a4788a6ab8f498093063166122"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.685501 17879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "4f3235b0d1154209aeb79ee7a561ff81"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "85ebf1abf284c97fefd0dba861324fdd"
server_key_iv: "a983b0a4788a6ab8f498093063166122"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.686870 17879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:10:43.687577 17895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.687749 17879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:10:43.687857 17879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "4f3235b0d1154209aeb79ee7a561ff81"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "85ebf1abf284c97fefd0dba861324fdd"
server_key_iv: "a983b0a4788a6ab8f498093063166122"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.687932 17879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:43.699556 17879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:43.701923 17879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:43.702041 17879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:43.702203 17879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:43.702466 17879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:43.702499 17879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.702521 17879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:43.702533 17879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.708143 17879 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:33607
I20260807 20:10:43.708236 18011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:33607 every 8 connection(s)
I20260807 20:10:43.708560 17879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:10:43.711540 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 17879
I20260807 20:10:43.711627 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:10:43.711843 3134 external_mini_cluster.cc:1428] Setting key afc1db81d8aee355c5faf1824b1865f7
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:43.718652 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.709325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46139 (local address 127.3.15.190:46207)
0807 20:10:43.709408 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.709410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.709606 (+ 196us) server_negotiation.cc:407] Connection header received
0807 20:10:43.710117 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.710118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.710130 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.710192 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:43.711170 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.711773 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.713109 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.713311 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.713397 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.715625 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:43.715627 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:43.715628 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:43.715639 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:43.717197 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.717655 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.717657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.717657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.717725 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.717982 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.717983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.717984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.718109 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:43.718181 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.718468 (+ 287us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.718556 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 20:10:43.718860 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.709274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:46139)
0807 20:10:43.709443 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:43.709485 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:43.710005 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:43.710273 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:43.710276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:43.710862 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:43.711056 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.711061 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.711878 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.711880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.713007 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.713010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.713409 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.713410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.713547 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.714130 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:43.714139 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:43.715524 (+ 1385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:43.717305 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:43.717313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:43.717315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:43.717559 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:43.717794 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:43.717795 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:43.717796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:43.717909 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:43.718183 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:43.718191 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:43.718333 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 20:10:43.718444 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 20:10:43.718617 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260807 20:10:43.719295 18012 heartbeater.cc:347] Connected to a master server at 127.3.15.190:46207
I20260807 20:10:43.719393 18012 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:43.719602 18012 heartbeater.cc:511] Master 127.3.15.190:46207 requested a full tablet report, sending...
I20260807 20:10:43.720028 17673 ts_manager.cc:194] Registered new tserver with Master: 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
I20260807 20:10:43.720409 17673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:46139
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:10:43.739564 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:46207
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:43.823962 18018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:43.824167 18018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:43.824189 18018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:43.825438 18018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:43.825480 18018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:43.825506 18018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:10:43.827047 18018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:43.827298 18018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:43.827535 18018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:43.832865 18027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:43.832876 18024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:43.832858 18023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:43.833119 18018 server_base.cc:1034] running on GCE node
I20260807 20:10:43.833325 18018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:43.833575 18018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:43.834721 18018 hybrid_clock.cc:648] HybridClock initialized: now 1786133443834703 us; error 35 us; skew 500 ppm
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:43.837415 18018 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:10:43.837929 18018 webserver.cc:460] Webserver started at http://127.3.15.131:41567/ using document root <none> and password file <none>
I20260807 20:10:43.838202 18018 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:43.838272 18018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:43.838411 18018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:10:43.839535 18018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "79c19e31e8504c0594262c39a1592266"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "a5e5ae2076f5fded5b4e9e5247341150"
server_key_iv: "fb6b38ecaa410cdfeec7169d3c91fab0"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.839905 18018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "79c19e31e8504c0594262c39a1592266"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "a5e5ae2076f5fded5b4e9e5247341150"
server_key_iv: "fb6b38ecaa410cdfeec7169d3c91fab0"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.841317 18018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:10:43.842024 18035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.842191 18018 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:10:43.842291 18018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "79c19e31e8504c0594262c39a1592266"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "a5e5ae2076f5fded5b4e9e5247341150"
server_key_iv: "fb6b38ecaa410cdfeec7169d3c91fab0"
server_key_version: "encryptionkey@0"
I20260807 20:10:43.842377 18018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:43.856891 18018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:43.858722 18018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:43.858894 18018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:43.859148 18018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:43.859464 18018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:43.859519 18018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.859560 18018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:43.859580 18018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:43.865593 18018 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:45267
I20260807 20:10:43.865636 18153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:45267 every 8 connection(s)
I20260807 20:10:43.865996 18018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:10:43 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133443, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:43.875401 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18018
I20260807 20:10:43.875524 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:10:43.875705 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.866761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:53151 (local address 127.3.15.190:46207)
0807 20:10:43.866833 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.866835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.867057 (+ 222us) server_negotiation.cc:407] Connection header received
0807 20:10:43.867539 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.867540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.867550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.867652 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:43.868911 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.869582 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.870652 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.870842 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.870898 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.873142 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:43.873144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:43.873144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:43.873151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:43.874398 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.874771 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.874772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.874773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.874824 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.875043 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.875044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.875044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.875162 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:43.875253 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.875509 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.875602 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:10:43.875732 3134 external_mini_cluster.cc:1428] Setting key 8fcf840a5cdfd7c77164b4786d1e3b7a
I20260807 20:10:43.875941 18156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.866759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:53151)
0807 20:10:43.866923 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:43.866966 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:43.867456 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:43.867780 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:43.867784 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:43.868468 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:43.868742 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.868750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.869671 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.869672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.870560 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.870563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.870917 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.870917 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.871021 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.871586 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:43.871605 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:43.873053 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:43.874472 (+ 1419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:43.874479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:43.874481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:43.874716 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:43.874886 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:43.874887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:43.874887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:43.874985 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:43.875241 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:43.875246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:43.875369 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 20:10:43.875453 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 20:10:43.875672 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260807 20:10:43.876631 18154 heartbeater.cc:347] Connected to a master server at 127.3.15.190:46207
I20260807 20:10:43.876747 18154 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:43.876955 18154 heartbeater.cc:511] Master 127.3.15.190:46207 requested a full tablet report, sending...
I20260807 20:10:43.877336 17673 ts_manager.cc:194] Registered new tserver with Master: 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267)
I20260807 20:10:43.877689 17673 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:53151
I20260807 20:10:43.887792 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:10:43.895432 17732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.889284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57484 (local address 127.3.15.190:46207)
0807 20:10:43.889338 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.889340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.889420 (+ 80us) server_negotiation.cc:407] Connection header received
0807 20:10:43.889478 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.889479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.889487 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.889532 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:43.890141 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.890673 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.891432 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.891611 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.891685 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.892993 (+ 1308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:43.892995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:43.892995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:43.893003 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:43.894236 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.894585 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.894586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.894587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.894651 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:43.894861 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:43.894862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:43.894862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:43.895015 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:43.895135 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.895252 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.895330 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:10:43.897243 17673 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:10:43.898865 17673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:10:43.905503 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.901510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:33607 (local address 127.0.0.1:45054)
0807 20:10:43.901842 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:43.901850 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:43.901883 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:43.902083 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:43.902087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:43.902089 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:43.902155 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.902157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.903092 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.903093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.904223 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.904226 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.905177 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.905178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.905258 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.905275 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 20:10:43.905343 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:10:43.905411 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":49,"threads_started":1}
I20260807 20:10:43.905608 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.901321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:45267 (local address 127.0.0.1:59144)
0807 20:10:43.901666 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:43.901675 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:43.901741 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:43.902245 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:43.902247 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:43.902249 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:43.902314 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.902315 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.903592 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.903593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.904536 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.904538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.905304 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.905305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.905381 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.905386 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:10:43.905446 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:10:43.905489 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":4,"thread_start_us":42,"threads_started":1}
I20260807 20:10:43.905759 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.901502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59144 (local address 127.3.15.131:45267)
0807 20:10:43.902018 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.902023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.902034 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:10:43.902073 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.902075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.902090 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.902159 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:43.902396 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.903486 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.904620 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.905229 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.905303 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.905360 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.905471 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.905509 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":87,"threads_started":1}
I20260807 20:10:43.905859 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.901646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.3.15.130:33607)
0807 20:10:43.901859 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.901863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.901875 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:10:43.901951 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.901953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.901968 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.902242 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:43.902322 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.902991 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.904312 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.905089 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.905164 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.905222 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.905724 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.905773 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":76,"threads_started":1}
I20260807 20:10:43.905869 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.901321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40817 (local address 127.0.0.1:39214)
0807 20:10:43.901621 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:43.901664 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:43.901741 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:43.902149 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:43.902150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:43.902151 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:43.902206 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.902207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.903544 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.903545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.904589 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:43.904593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.905613 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.905614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:43.905688 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.905693 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:10:43.905748 (+ 55us) client_negotiation.cc:240] Negotiation successful
0807 20:10:43.905788 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":48,"threads_started":1}
I20260807 20:10:43.906091 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:43.901460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39214 (local address 127.3.15.129:40817)
0807 20:10:43.901717 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:43.901723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:43.901737 (+ 14us) server_negotiation.cc:407] Connection header received
0807 20:10:43.901858 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:43.901862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:43.901879 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:43.901979 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:43.902270 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.903401 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.904720 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:43.905466 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:43.905526 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:43.905579 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:43.905946 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 20:10:43.905999 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":122,"threads_started":1}
I20260807 20:10:43.906608 18085 tablet_service.cc:1467] Processing CreateTablet for tablet d6d1011ff73540f1a5a70624ff10a500 (DEFAULT_TABLE table=test-table [id=9d6bc1ef0097410cb0998e4e5f74e492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:43.906571 17944 tablet_service.cc:1467] Processing CreateTablet for tablet d6d1011ff73540f1a5a70624ff10a500 (DEFAULT_TABLE table=test-table [id=9d6bc1ef0097410cb0998e4e5f74e492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:43.906955 17944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6d1011ff73540f1a5a70624ff10a500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:43.906955 18085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6d1011ff73540f1a5a70624ff10a500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:43.910437 18172 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Bootstrap starting.
I20260807 20:10:43.910454 18171 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Bootstrap starting.
I20260807 20:10:43.910941 17806 tablet_service.cc:1467] Processing CreateTablet for tablet d6d1011ff73540f1a5a70624ff10a500 (DEFAULT_TABLE table=test-table [id=9d6bc1ef0097410cb0998e4e5f74e492]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:43.911228 18171 tablet_bootstrap.cc:654] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:43.911228 18172 tablet_bootstrap.cc:654] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:43.911293 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6d1011ff73540f1a5a70624ff10a500. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:43.911902 18171 log.cc:824] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:43.911893 18172 log.cc:824] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:43.912951 18171 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: No bootstrap required, opened a new log
I20260807 20:10:43.913034 18171 ts_tablet_manager.cc:1397] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:10:43.913058 18172 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: No bootstrap required, opened a new log
I20260807 20:10:43.913143 18172 ts_tablet_manager.cc:1397] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:10:43.914522 18176 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap starting.
I20260807 20:10:43.915174 18176 tablet_bootstrap.cc:654] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:43.915081 18171 raft_consensus.cc:348] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:43.915081 18172 raft_consensus.cc:348] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:43.915266 18172 raft_consensus.cc:374] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:43.915266 18171 raft_consensus.cc:374] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:43.915313 18171 raft_consensus.cc:729] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c19e31e8504c0594262c39a1592266, State: Initialized, Role: FOLLOWER
I20260807 20:10:43.915313 18172 raft_consensus.cc:729] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3235b0d1154209aeb79ee7a561ff81, State: Initialized, Role: FOLLOWER
I20260807 20:10:43.915453 18171 consensus_queue.cc:260] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:43.915453 18172 consensus_queue.cc:260] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:43.915772 18172 ts_tablet_manager.cc:1428] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:10:43.915772 18171 ts_tablet_manager.cc:1428] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:10:43.915856 18176 log.cc:824] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:43.915966 18012 heartbeater.cc:503] Master 127.3.15.190:46207 was elected leader, sending a full tablet report...
I20260807 20:10:43.916028 18154 heartbeater.cc:503] Master 127.3.15.190:46207 was elected leader, sending a full tablet report...
I20260807 20:10:43.916843 18176 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: No bootstrap required, opened a new log
I20260807 20:10:43.916923 18176 ts_tablet_manager.cc:1397] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:10:43.918426 18176 raft_consensus.cc:348] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:43.918570 18176 raft_consensus.cc:374] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:43.918596 18176 raft_consensus.cc:729] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8184af9eee254c2b99a8f04d39d56a70, State: Initialized, Role: FOLLOWER
I20260807 20:10:43.918712 18176 consensus_queue.cc:260] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:43.918943 18176 ts_tablet_manager.cc:1428] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:10:43.919064 17873 heartbeater.cc:503] Master 127.3.15.190:46207 was elected leader, sending a full tablet report...
W20260807 20:10:43.959224 18013 tablet.cc:2369] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:10:44.050684 17874 tablet.cc:2369] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:10:44.116640 18155 tablet.cc:2369] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:10:44.178135 18177 raft_consensus.cc:482] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:44.178284 18177 raft_consensus.cc:504] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:44.178629 18177 leader_election.cc:288] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817)
I20260807 20:10:44.181620 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.178844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40817 (local address 127.3.15.130:45239)
0807 20:10:44.178944 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.178960 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.179033 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.179197 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.179198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.179199 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:44.179284 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.179286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.179963 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.179964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.180727 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.180729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.181345 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.181346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.181435 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.181438 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.181506 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.181546 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":1}
I20260807 20:10:44.181617 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.178982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:45239 (local address 127.3.15.129:40817)
0807 20:10:44.179037 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.179040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.179051 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:10:44.179106 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.179107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.179117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.179191 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:44.179359 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.179862 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.180903 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.181263 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.181349 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.181358 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.181511 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.181549 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:10:44.181797 18181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.178844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:45267 (local address 127.3.15.130:37149)
0807 20:10:44.179158 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.179169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.179256 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.179401 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.179401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.179403 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:44.179478 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.179480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.180131 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.180132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.180960 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.180962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.181510 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.181511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.181587 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.181599 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.181673 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.181714 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":88,"thread_start_us":180,"threads_started":1}
I20260807 20:10:44.181783 18169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.178976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:37149 (local address 127.3.15.131:45267)
0807 20:10:44.179028 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.179031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.179245 (+ 214us) server_negotiation.cc:407] Connection header received
0807 20:10:44.179317 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.179318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.179327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.179398 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:44.179556 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.180025 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.181047 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.181422 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.181509 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.181518 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.181679 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.181716 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:10:44.181991 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d1011ff73540f1a5a70624ff10a500" candidate_uuid: "4f3235b0d1154209aeb79ee7a561ff81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8184af9eee254c2b99a8f04d39d56a70" is_pre_election: true
I20260807 20:10:44.182126 18106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d1011ff73540f1a5a70624ff10a500" candidate_uuid: "4f3235b0d1154209aeb79ee7a561ff81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c19e31e8504c0594262c39a1592266" is_pre_election: true
I20260807 20:10:44.182163 17826 raft_consensus.cc:2454] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3235b0d1154209aeb79ee7a561ff81 in term 0.
I20260807 20:10:44.182262 18106 raft_consensus.cc:2454] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3235b0d1154209aeb79ee7a561ff81 in term 0.
I20260807 20:10:44.182427 17899 leader_election.cc:302] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3235b0d1154209aeb79ee7a561ff81, 8184af9eee254c2b99a8f04d39d56a70; no voters:
I20260807 20:10:44.182578 18177 raft_consensus.cc:2781] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:10:44.182641 18177 raft_consensus.cc:482] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:10:44.182672 18177 raft_consensus.cc:3037] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:44.183389 18177 raft_consensus.cc:504] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:44.183528 18177 leader_election.cc:288] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [CANDIDATE]: Term 1 election: Requested vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817)
I20260807 20:10:44.183701 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d1011ff73540f1a5a70624ff10a500" candidate_uuid: "4f3235b0d1154209aeb79ee7a561ff81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8184af9eee254c2b99a8f04d39d56a70"
I20260807 20:10:44.183699 18106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6d1011ff73540f1a5a70624ff10a500" candidate_uuid: "4f3235b0d1154209aeb79ee7a561ff81" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c19e31e8504c0594262c39a1592266"
I20260807 20:10:44.183777 18106 raft_consensus.cc:3037] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:44.183777 17826 raft_consensus.cc:3037] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:44.184689 17826 raft_consensus.cc:2454] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3235b0d1154209aeb79ee7a561ff81 in term 1.
I20260807 20:10:44.184689 18106 raft_consensus.cc:2454] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3235b0d1154209aeb79ee7a561ff81 in term 1.
I20260807 20:10:44.184952 17896 leader_election.cc:302] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3235b0d1154209aeb79ee7a561ff81, 79c19e31e8504c0594262c39a1592266; no voters:
I20260807 20:10:44.185035 18177 raft_consensus.cc:2781] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:44.185220 18177 raft_consensus.cc:686] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 LEADER]: Becoming Leader. State: Replica: 4f3235b0d1154209aeb79ee7a561ff81, State: Running, Role: LEADER
I20260807 20:10:44.185324 18177 consensus_queue.cc:237] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:44.186124 17673 catalog_manager.cc:5495] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "4f3235b0d1154209aeb79ee7a561ff81" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } health_report { overall_health: HEALTHY } } }
I20260807 20:10:44.258726 18170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.256392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.3.15.130:33607)
0807 20:10:44.256481 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.256482 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.256506 (+ 24us) server_negotiation.cc:407] Connection header received
0807 20:10:44.256546 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.256547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.256568 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.256657 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:10:44.256803 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.257268 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.258070 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.258257 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.258307 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.258442 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:10:44.258545 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:10:44.258624 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.258638 (+ 14us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.258655 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 20:10:44.260653 18106 raft_consensus.cc:1261] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Refusing update from remote peer 4f3235b0d1154209aeb79ee7a561ff81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:44.260682 17826 raft_consensus.cc:1261] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Refusing update from remote peer 4f3235b0d1154209aeb79ee7a561ff81: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:44.261056 18182 consensus_queue.cc:1035] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:44.261193 18177 consensus_queue.cc:1035] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:44.263563 18185 mvcc.cc:205] Tried to move back new op lower bound from 7316002587688497152 to 7316002587383791616. Current Snapshot: MvccSnapshot[applied={T|T < 7316002587688497152}]
I20260807 20:10:44.264632 18186 mvcc.cc:205] Tried to move back new op lower bound from 7316002587688497152 to 7316002587383791616. Current Snapshot: MvccSnapshot[applied={T|T < 7316002587688497152}]
I20260807 20:10:44.265090 18187 mvcc.cc:205] Tried to move back new op lower bound from 7316002587688497152 to 7316002587383791616. Current Snapshot: MvccSnapshot[applied={T|T < 7316002587688497152 or (T in {7316002587688497152})}]
I20260807 20:10:44.266079 17672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:10:44.266179 17672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:10:44.267035 17672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Sending DeleteTablet for 3 replicas of tablet d6d1011ff73540f1a5a70624ff10a500
I20260807 20:10:44.267408 18085 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d1011ff73540f1a5a70624ff10a500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59144
I20260807 20:10:44.267423 17806 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d1011ff73540f1a5a70624ff10a500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:39214
I20260807 20:10:44.267688 18197 tablet_replica.cc:330] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: stopping tablet replica
I20260807 20:10:44.267777 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17737
I20260807 20:10:44.267771 17944 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d1011ff73540f1a5a70624ff10a500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:45054
I20260807 20:10:44.267943 18197 raft_consensus.cc:2229] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:10:44.268100 18197 raft_consensus.cc:2258] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.269502 18198 tablet_replica.cc:330] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: stopping tablet replica
I20260807 20:10:44.269641 18198 raft_consensus.cc:2229] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:10:44.269761 18198 raft_consensus.cc:2258] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.271122 18198 ts_tablet_manager.cc:1905] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:44.273538 18198 ts_tablet_manager.cc:1918] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:44.273629 18198 log.cc:1196] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/d6d1011ff73540f1a5a70624ff10a500
I20260807 20:10:44.273914 18198 ts_tablet_manager.cc:1939] T d6d1011ff73540f1a5a70624ff10a500 P 4f3235b0d1154209aeb79ee7a561ff81: Deleting consensus metadata
I20260807 20:10:44.274385 17656 catalog_manager.cc:4841] TS 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): tablet d6d1011ff73540f1a5a70624ff10a500 (table test-table [id=9d6bc1ef0097410cb0998e4e5f74e492]) successfully deleted
I20260807 20:10:44.277695 18196 tablet_replica.cc:330] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: stopping tablet replica
W20260807 20:10:44.277707 17659 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:40817 (error 108)
I20260807 20:10:44.277837 18196 raft_consensus.cc:2229] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:10:44.277976 18196 raft_consensus.cc:2258] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.278275 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.278131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40817 (local address 127.0.0.1:39222)
0807 20:10:44.278188 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.278232 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:40817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 20:10:44.278409 17659 catalog_manager.cc:4584] TS 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d6d1011ff73540f1a5a70624ff10a500: Network error: Client connection negotiation failed: client connection to 127.3.15.129:40817: connect: Connection refused (error 111)
I20260807 20:10:44.278515 18196 ts_tablet_manager.cc:1905] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:44.278805 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17879
I20260807 20:10:44.280931 18196 ts_tablet_manager.cc:1918] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:44.281028 18196 log.cc:1196] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/d6d1011ff73540f1a5a70624ff10a500
I20260807 20:10:44.281343 18196 ts_tablet_manager.cc:1939] T d6d1011ff73540f1a5a70624ff10a500 P 79c19e31e8504c0594262c39a1592266: Deleting consensus metadata
I20260807 20:10:44.281802 17656 catalog_manager.cc:4841] TS 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): tablet d6d1011ff73540f1a5a70624ff10a500 (table test-table [id=9d6bc1ef0097410cb0998e4e5f74e492]) successfully deleted
I20260807 20:10:44.284643 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18018
I20260807 20:10:44.291488 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 17640
I20260807 20:10:44.298629 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:46207
--webserver_interface=127.3.15.190
--webserver_port=36353
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:46207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:44.382833 18200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:44.383073 18200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:44.383097 18200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:44.384311 18200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:44.384358 18200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:44.384372 18200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:44.384385 18200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:44.384397 18200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:44.386058 18200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:46207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:46207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=36353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:44.386344 18200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:44.386595 18200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:10:44.396456 18200 server_base.cc:1034] running on GCE node
W20260807 20:10:44.396409 18207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:44.396416 18206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:44.396436 18209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:44.397032 18200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:44.397365 18200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:44.398536 18200 hybrid_clock.cc:648] HybridClock initialized: now 1786133444398524 us; error 27 us; skew 500 ppm
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:44.401110 18200 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:44.401577 18200 webserver.cc:460] Webserver started at http://127.3.15.190:36353/ using document root <none> and password file <none>
I20260807 20:10:44.401863 18200 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:44.401953 18200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:44.403400 18200 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:44.404090 18216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.404263 18200 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:10:44.404376 18200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "e593bd2e00a14db0b624be17e624d48d"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "86203971a2c5c58d15c69a732c82aacb"
server_key_iv: "d929835e16643c5eb2a6ad8d6305eb3a"
server_key_version: "encryptionkey@0"
I20260807 20:10:44.404862 18200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:44.414121 18200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:44.416571 18200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:44.416749 18200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:44.420876 18200 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:46207
I20260807 20:10:44.420905 18271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:46207 every 8 connection(s)
I20260807 20:10:44.421339 18200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:44.423316 18272 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:10:44.424084 18272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap starting.
I20260807 20:10:44.424383 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18200
I20260807 20:10:44.424686 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:40817
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:46207
--webserver_port=40453
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:44.433476 18272 log.cc:824] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:44.435729 18272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:10:44.436096 18272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap complete.
I20260807 20:10:44.438735 18272 raft_consensus.cc:348] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:44.439016 18272 raft_consensus.cc:729] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e593bd2e00a14db0b624be17e624d48d, State: Initialized, Role: FOLLOWER
I20260807 20:10:44.439225 18272 consensus_queue.cc:260] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:44.439314 18272 raft_consensus.cc:388] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:10:44.439383 18272 raft_consensus.cc:482] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:10:44.439443 18272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:10:44.440577 18272 raft_consensus.cc:504] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:44.440708 18272 leader_election.cc:302] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e593bd2e00a14db0b624be17e624d48d; no voters:
I20260807 20:10:44.440982 18272 leader_election.cc:288] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:10:44.441022 18277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:10:44.441143 18277 raft_consensus.cc:686] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 LEADER]: Becoming Leader. State: Replica: e593bd2e00a14db0b624be17e624d48d, State: Running, Role: LEADER
I20260807 20:10:44.441250 18277 consensus_queue.cc:237] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:44.441299 18272 sys_catalog.cc:564] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:44.441560 18277 sys_catalog.cc:455] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e593bd2e00a14db0b624be17e624d48d. Latest consensus state: current_term: 2 leader_uuid: "e593bd2e00a14db0b624be17e624d48d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } } }
I20260807 20:10:44.441636 18277 sys_catalog.cc:458] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:44.441783 18278 sys_catalog.cc:455] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e593bd2e00a14db0b624be17e624d48d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } } }
I20260807 20:10:44.441862 18278 sys_catalog.cc:458] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:44.442202 18281 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:44.443162 18281 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:44.443284 18281 catalog_manager.cc:1259] Loaded cluster ID: 32071f15414a4e8ebddfc26fa7fa8cfd
I20260807 20:10:44.443327 18281 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:10:44.443755 18281 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 20:10:44.446936 18281 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 20:10:44.449880 18281 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:10:44.450098 18281 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Loaded TSK: 0
I20260807 20:10:44.450338 18281 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:10:44.522576 18274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:44.522794 18274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:44.522819 18274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:44.524049 18274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:44.524096 18274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:44.524147 18274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:10:44.526371 18274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:40817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=40453
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:44.526603 18274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:44.526954 18274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:44.533192 18301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:44.533162 18300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:44.533192 18303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:44.533468 18274 server_base.cc:1034] running on GCE node
I20260807 20:10:44.533692 18274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:44.533998 18274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:44.535173 18274 hybrid_clock.cc:648] HybridClock initialized: now 1786133444535158 us; error 40 us; skew 500 ppm
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:44.538280 18274 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:10:44.538894 18274 webserver.cc:460] Webserver started at http://127.3.15.129:40453/ using document root <none> and password file <none>
I20260807 20:10:44.539178 18274 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:44.539274 18274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:44.541028 18274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:10:44.541994 18310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.542276 18274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:44.542387 18274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "8184af9eee254c2b99a8f04d39d56a70"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "7f0c1f351d415401edfd78406d2e2fc0"
server_key_iv: "9a5bad36fb075568f43a6d878455571a"
server_key_version: "encryptionkey@0"
I20260807 20:10:44.542887 18274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:44.575146 18274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:44.577843 18274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:44.578002 18274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:44.578260 18274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:44.578742 18317 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:10:44.580293 18274 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:10:44.580354 18274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:10:44.580389 18274 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:10:44.581287 18274 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:10:44.581332 18274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:44.581418 18317 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap starting.
I20260807 20:10:44.588644 18274 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40817
I20260807 20:10:44.588867 18428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40817 every 8 connection(s)
I20260807 20:10:44.589174 18274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:10:44.591310 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18274
I20260807 20:10:44.591665 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:33607
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:46207
--webserver_port=35341
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:44.592525 18317 log.cc:824] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:44.595376 18317 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:10:44.595706 18317 tablet_bootstrap.cc:492] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap complete.
I20260807 20:10:44.596951 18317 ts_tablet_manager.cc:1397] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.015s
I20260807 20:10:44.598850 18317 raft_consensus.cc:348] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:44.599256 18317 raft_consensus.cc:729] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8184af9eee254c2b99a8f04d39d56a70, State: Initialized, Role: FOLLOWER
I20260807 20:10:44.599428 18317 consensus_queue.cc:260] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } }
I20260807 20:10:44.599756 18317 ts_tablet_manager.cc:1428] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:44.606532 18431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.590017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:36709)
0807 20:10:44.590251 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.590309 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.591163 (+ 854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.595174 (+ 4011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.595180 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.595552 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:44.595831 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.595838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.597552 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.597553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.598878 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.598883 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.599699 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.599700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.599827 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.600493 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:44.600503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:44.602397 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:44.604826 (+ 2429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.604834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.604836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.605071 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.605370 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.605371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.605371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.605469 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.606020 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:44.606028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:44.606144 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.606234 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.606359 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":122,"threads_started":1}
I20260807 20:10:44.606668 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.590355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:36709 (local address 127.3.15.190:46207)
0807 20:10:44.592872 (+ 2517us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.592882 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.592929 (+ 47us) server_negotiation.cc:407] Connection header received
0807 20:10:44.594849 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.594860 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.594947 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.595108 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:44.595979 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.597427 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.599016 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.599591 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.600128 (+ 537us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.602526 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:44.602531 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:44.602531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:44.602545 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:44.604681 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.605163 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.605166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.605167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.605292 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.605545 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.605547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.605547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.605933 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:44.606037 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.606330 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.606481 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2462,"thread_start_us":95,"threads_started":1}
I20260807 20:10:44.607296 18429 heartbeater.cc:347] Connected to a master server at 127.3.15.190:46207
I20260807 20:10:44.607414 18429 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:44.607617 18429 heartbeater.cc:511] Master 127.3.15.190:46207 requested a full tablet report, sending...
I20260807 20:10:44.608323 18234 ts_manager.cc:194] Registered new tserver with Master: 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817)
I20260807 20:10:44.609429 18234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:36709
I20260807 20:10:44.610602 18429 heartbeater.cc:503] Master 127.3.15.190:46207 was elected leader, sending a full tablet report...
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.129@KRBTEST.COM
I20260807 20:10:44.617969 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.608900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40817 (local address 127.0.0.1:39234)
0807 20:10:44.609089 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.609125 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.609188 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.609464 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.609471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.609782 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:44.609836 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.609838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.611603 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.611604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.612526 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.612528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.613046 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.613048 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.613141 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.613578 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:44.613585 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:44.614964 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:44.616689 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.616697 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.616699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.616971 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.617213 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.617214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.617215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.617277 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.617573 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:44.617581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:44.617633 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.617728 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.617874 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":104,"threads_started":1}
I20260807 20:10:44.617969 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.609160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.3.15.129:40817)
0807 20:10:44.609307 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.609311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.609320 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:10:44.609351 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.609353 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.609367 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.609432 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:44.609927 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.611526 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.612614 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.612961 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.613025 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.615070 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:44.615079 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:44.615079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:44.615086 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:44.616565 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.617072 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.617075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.617075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.617142 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.617350 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.617351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.617352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.617495 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:44.617581 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.617778 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.617871 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260807 20:10:44.618583 18361 tablet_service.cc:1514] Processing DeleteTablet for tablet d6d1011ff73540f1a5a70624ff10a500 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:39234
I20260807 20:10:44.619089 18438 tablet_replica.cc:330] stopping tablet replica
I20260807 20:10:44.619199 18438 raft_consensus.cc:2229] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:10:44.619268 18438 raft_consensus.cc:2258] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.619589 18438 ts_tablet_manager.cc:1905] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:44.622057 18438 ts_tablet_manager.cc:1918] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:44.622136 18438 log.cc:1196] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/d6d1011ff73540f1a5a70624ff10a500
I20260807 20:10:44.622360 18438 ts_tablet_manager.cc:1939] T d6d1011ff73540f1a5a70624ff10a500 P 8184af9eee254c2b99a8f04d39d56a70: Deleting consensus metadata
I20260807 20:10:44.622876 18220 catalog_manager.cc:4841] TS 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817): tablet d6d1011ff73540f1a5a70624ff10a500 (table test-table [id=9d6bc1ef0097410cb0998e4e5f74e492]) successfully deleted
W20260807 20:10:44.684680 18434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:44.684921 18434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:44.684954 18434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:44.686061 18434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:44.686095 18434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:44.686120 18434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:10:44.687500 18434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:33607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=35341
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:44.687776 18434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:44.687985 18434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:44.693058 18443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:44.693257 18434 server_base.cc:1034] running on GCE node
W20260807 20:10:44.693071 18446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:44.693087 18444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:44.693569 18434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:44.693826 18434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:44.694993 18434 hybrid_clock.cc:648] HybridClock initialized: now 1786133444694977 us; error 43 us; skew 500 ppm
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:44.697510 18434 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:10:44.697888 18434 webserver.cc:460] Webserver started at http://127.3.15.130:35341/ using document root <none> and password file <none>
I20260807 20:10:44.698067 18434 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:44.698118 18434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:44.699429 18434 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:44.700006 18454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.700158 18434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:10:44.700242 18434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "4f3235b0d1154209aeb79ee7a561ff81"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "85ebf1abf284c97fefd0dba861324fdd"
server_key_iv: "a983b0a4788a6ab8f498093063166122"
server_key_version: "encryptionkey@0"
I20260807 20:10:44.700577 18434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:44.716367 18434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:44.718716 18434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:44.718827 18434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:44.719022 18434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:44.719278 18434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:44.719308 18434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.719331 18434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:44.719343 18434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.724970 18434 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:33607
I20260807 20:10:44.725033 18573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:33607 every 8 connection(s)
I20260807 20:10:44.725360 18434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:10:44.729534 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18434
I20260807 20:10:44.729833 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:45267
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:46207
--webserver_port=41567
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:44.735477 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.726116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56805 (local address 127.3.15.190:46207)
0807 20:10:44.726187 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.726189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.726402 (+ 213us) server_negotiation.cc:407] Connection header received
0807 20:10:44.726898 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.726899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.726909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.726968 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:44.727907 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.728411 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.729552 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.729922 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.730001 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.732302 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:44.732304 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:44.732304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:44.732319 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:44.733815 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.734391 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.734393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.734393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.734475 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.734791 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.734791 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.734792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.734905 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:44.734961 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.735327 (+ 366us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.735401 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:10:44.735790 18576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.726096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:56805)
0807 20:10:44.726275 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.726317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.726810 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.727048 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.727051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.727634 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:44.727827 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.727832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.728506 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.728507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.729455 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.729458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.730014 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.730016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.730165 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.730693 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:44.730701 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:44.732196 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:44.733907 (+ 1711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.733916 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.733918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.734298 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.734552 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.734553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.734554 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.734714 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.734976 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:44.734984 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:44.735177 (+ 193us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.735309 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.735502 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":108,"threads_started":1}
I20260807 20:10:44.736390 18574 heartbeater.cc:347] Connected to a master server at 127.3.15.190:46207
I20260807 20:10:44.736495 18574 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:44.736706 18574 heartbeater.cc:511] Master 127.3.15.190:46207 requested a full tablet report, sending...
I20260807 20:10:44.737161 18234 ts_manager.cc:194] Registered new tserver with Master: 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
I20260807 20:10:44.737540 18234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:56805
W20260807 20:10:44.815042 18577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:44.815285 18577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:44.815322 18577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:44.816464 18577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:44.816530 18577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:44.816571 18577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:10:44.818086 18577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:45267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=41567
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:44.818356 18577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:44.818599 18577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:44.823870 18583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:44.823895 18582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:44.824000 18577 server_base.cc:1034] running on GCE node
W20260807 20:10:44.823868 18585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:44.824265 18577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:44.824467 18577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:44.825603 18577 hybrid_clock.cc:648] HybridClock initialized: now 1786133444825587 us; error 33 us; skew 500 ppm
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:44.828356 18577 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:10:44.829007 18577 webserver.cc:460] Webserver started at http://127.3.15.131:41567/ using document root <none> and password file <none>
I20260807 20:10:44.829267 18577 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:44.829329 18577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:44.830708 18577 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:10:44.837667 18592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.838160 18577 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260807 20:10:44.838269 18577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "79c19e31e8504c0594262c39a1592266"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "a5e5ae2076f5fded5b4e9e5247341150"
server_key_iv: "fb6b38ecaa410cdfeec7169d3c91fab0"
server_key_version: "encryptionkey@0"
I20260807 20:10:44.838675 18577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:44.851259 18577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:44.853647 18577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:44.853812 18577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:44.854074 18577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:44.854386 18577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:44.854442 18577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.854480 18577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:44.854525 18577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:44.860541 18577 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:45267
I20260807 20:10:44.860597 18711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:45267 every 8 connection(s)
I20260807 20:10:44.861037 18577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:10:44 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133444, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:10:44.868037 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18577
I20260807 20:10:44.871387 18432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.861786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:34175 (local address 127.3.15.190:46207)
0807 20:10:44.861852 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.861854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.862077 (+ 223us) server_negotiation.cc:407] Connection header received
0807 20:10:44.862541 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.862542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.862552 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.862611 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:44.863570 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.864088 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.865117 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.865301 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.865356 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.867998 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:44.868000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:44.868000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:44.868008 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:44.869762 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.870207 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.870209 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.870209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.870276 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.870602 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.870604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.870604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.870744 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:44.870812 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.871191 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.871289 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:10:44.871587 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.861784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:34175)
0807 20:10:44.861941 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.861982 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.862457 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.862662 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.862665 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.863251 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:44.863468 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.863473 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.864169 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.864170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.865042 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.865045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.865372 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.865372 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.865471 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.866017 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:44.866025 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:44.867869 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:44.869840 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.869847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.869849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.870112 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.870356 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:44.870357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:44.870358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:44.870496 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:44.870819 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:44.870827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:44.871016 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.871151 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.871336 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260807 20:10:44.872267 18712 heartbeater.cc:347] Connected to a master server at 127.3.15.190:46207
I20260807 20:10:44.872375 18712 heartbeater.cc:464] Registering TS with master...
I20260807 20:10:44.872587 18712 heartbeater.cc:511] Master 127.3.15.190:46207 requested a full tablet report, sending...
I20260807 20:10:44.872999 18234 ts_manager.cc:194] Registered new tserver with Master: 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267)
I20260807 20:10:44.873631 18234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:34175
I20260807 20:10:44.877401 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.868455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57498 (local address 127.3.15.190:46207)
0807 20:10:44.868572 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.868574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.868606 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:10:44.868679 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.868680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.868694 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.868766 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:44.869526 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.870338 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.872080 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.872381 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.872451 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.874096 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:44.874097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:44.874098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:44.874106 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:44.876026 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.876462 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.876463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.876463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.876532 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:44.876761 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:44.876762 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:44.876762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:44.876946 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:44.877130 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.877215 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.877305 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":64,"threads_started":1}
I20260807 20:10:44.878147 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:10:44.880959 18716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.878481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57514 (local address 127.3.15.190:46207)
0807 20:10:44.878541 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.878543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.878594 (+ 51us) server_negotiation.cc:407] Connection header received
0807 20:10:44.878679 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.878680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.878716 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.878800 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:10:44.878923 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.879392 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.880129 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.880340 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.880425 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.880611 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:10:44.880712 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:10:44.880818 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.880856 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.880879 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:10:44.881247 18234 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57514:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:10:44.882838 18234 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:10:44.885119 18361 tablet_service.cc:1467] Processing CreateTablet for tablet 4f07573f580f4383a2b623eb3be01268 (DEFAULT_TABLE table=test-table [id=fbe9658196ab457caa88e02b205f7229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:44.885341 18361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f07573f580f4383a2b623eb3be01268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:44.887019 18317 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap starting.
I20260807 20:10:44.887362 18317 tablet_bootstrap.cc:654] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:44.888479 18317 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: No bootstrap required, opened a new log
I20260807 20:10:44.888540 18317 ts_tablet_manager.cc:1397] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:10:44.888686 18317 raft_consensus.cc:348] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.888767 18317 raft_consensus.cc:374] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:44.888873 18317 raft_consensus.cc:729] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8184af9eee254c2b99a8f04d39d56a70, State: Initialized, Role: FOLLOWER
I20260807 20:10:44.888924 18317 consensus_queue.cc:260] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.889046 18317 ts_tablet_manager.cc:1428] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 20:10:44.889331 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.884764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:45267 (local address 127.0.0.1:59158)
0807 20:10:44.884832 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.884844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.884887 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.885555 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.885556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.885558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:44.885596 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.885598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.886826 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.886827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.887872 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.887875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.889002 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.889003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.889080 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.889104 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.889176 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.889224 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 20:10:44.889360 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.885121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.3.15.131:45267)
0807 20:10:44.885372 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.885377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.885389 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:10:44.885431 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.885433 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.885449 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.885530 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:44.885692 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.886724 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.887976 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.888887 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.889032 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.889108 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.889216 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.889258 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":91,"threads_started":1}
I20260807 20:10:44.890292 18643 tablet_service.cc:1467] Processing CreateTablet for tablet 4f07573f580f4383a2b623eb3be01268 (DEFAULT_TABLE table=test-table [id=fbe9658196ab457caa88e02b205f7229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:44.890707 18643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f07573f580f4383a2b623eb3be01268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:44.891067 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.885042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45078 (local address 127.3.15.130:33607)
0807 20:10:44.887115 (+ 2073us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.887120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.887132 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:10:44.887178 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.887180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.887198 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.887288 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:44.887427 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.888510 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.889868 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.890710 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.890767 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.890833 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.890979 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.891009 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2034,"thread_start_us":89,"threads_started":1}
I20260807 20:10:44.891065 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.884936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:33607 (local address 127.0.0.1:45078)
0807 20:10:44.885099 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.885109 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.885164 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.887297 (+ 2133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.887298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.887301 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:44.887341 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.887342 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.888648 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.888649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.889785 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.889788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.890823 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.890824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.890894 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.890898 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.890960 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.891000 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260807 20:10:44.891942 18505 tablet_service.cc:1467] Processing CreateTablet for tablet 4f07573f580f4383a2b623eb3be01268 (DEFAULT_TABLE table=test-table [id=fbe9658196ab457caa88e02b205f7229]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:10:44.892297 18505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f07573f580f4383a2b623eb3be01268. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:10:44.894388 18723 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Bootstrap starting.
I20260807 20:10:44.894972 18723 tablet_bootstrap.cc:654] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:44.895530 18723 log.cc:824] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:44.895680 18724 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Bootstrap starting.
I20260807 20:10:44.896329 18724 tablet_bootstrap.cc:654] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Neither blocks nor log segments found. Creating new log.
I20260807 20:10:44.896425 18723 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: No bootstrap required, opened a new log
I20260807 20:10:44.896502 18723 ts_tablet_manager.cc:1397] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:10:44.897079 18724 log.cc:824] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:44.897878 18724 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: No bootstrap required, opened a new log
I20260807 20:10:44.897956 18724 ts_tablet_manager.cc:1397] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:10:44.898623 18723 raft_consensus.cc:348] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.898813 18723 raft_consensus.cc:374] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:44.898854 18723 raft_consensus.cc:729] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c19e31e8504c0594262c39a1592266, State: Initialized, Role: FOLLOWER
I20260807 20:10:44.898977 18723 consensus_queue.cc:260] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.899271 18723 ts_tablet_manager.cc:1428] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:10:44.899308 18712 heartbeater.cc:503] Master 127.3.15.190:46207 was elected leader, sending a full tablet report...
I20260807 20:10:44.899971 18724 raft_consensus.cc:348] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.900148 18724 raft_consensus.cc:374] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:10:44.900199 18724 raft_consensus.cc:729] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3235b0d1154209aeb79ee7a561ff81, State: Initialized, Role: FOLLOWER
I20260807 20:10:44.900309 18724 consensus_queue.cc:260] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.900607 18574 heartbeater.cc:503] Master 127.3.15.190:46207 was elected leader, sending a full tablet report...
I20260807 20:10:44.900614 18724 ts_tablet_manager.cc:1428] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:10:44.964051 18727 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:44.964193 18727 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.964540 18727 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607), 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817)
I20260807 20:10:44.968604 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.964870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:49891 (local address 127.3.15.129:40817)
0807 20:10:44.964921 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.964923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.964943 (+ 20us) server_negotiation.cc:407] Connection header received
0807 20:10:44.964974 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.964975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.965006 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.965081 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:44.965302 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.966091 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.967639 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.968271 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.968339 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.968423 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.968487 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.968531 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 20:10:44.968600 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.964935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:53351 (local address 127.3.15.130:33607)
0807 20:10:44.964994 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.964997 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.965010 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:10:44.965067 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.965069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.965081 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.965134 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:10:44.965299 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.966057 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.967639 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.968215 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.968302 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.968314 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.968489 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.968530 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 20:10:44.968667 18714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.964736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40817 (local address 127.3.15.131:49891)
0807 20:10:44.964814 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.964827 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.964887 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.965110 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.965111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.965113 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:44.965224 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.965226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.966160 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.966161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.967533 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.967536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.968339 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.968340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.968401 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.968403 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.968478 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.968516 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":92}
I20260807 20:10:44.968679 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.964750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:33607 (local address 127.3.15.131:53351)
0807 20:10:44.964945 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.964964 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:44.965020 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:44.965134 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:44.965135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:44.965136 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:10:44.965224 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.965226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.966160 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.966161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.967533 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:44.967536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.968320 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.968321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:44.968398 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.968403 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:10:44.968459 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:10:44.968504 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":77,"thread_start_us":118,"threads_started":1}
I20260807 20:10:44.969087 18526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f07573f580f4383a2b623eb3be01268" candidate_uuid: "79c19e31e8504c0594262c39a1592266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3235b0d1154209aeb79ee7a561ff81" is_pre_election: true
I20260807 20:10:44.969105 18382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f07573f580f4383a2b623eb3be01268" candidate_uuid: "79c19e31e8504c0594262c39a1592266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8184af9eee254c2b99a8f04d39d56a70" is_pre_election: true
I20260807 20:10:44.969278 18526 raft_consensus.cc:2454] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c19e31e8504c0594262c39a1592266 in term 0.
I20260807 20:10:44.969283 18382 raft_consensus.cc:2454] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79c19e31e8504c0594262c39a1592266 in term 0.
I20260807 20:10:44.969538 18596 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c19e31e8504c0594262c39a1592266, 8184af9eee254c2b99a8f04d39d56a70; no voters:
I20260807 20:10:44.969710 18727 raft_consensus.cc:2781] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:10:44.969771 18727 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:10:44.969794 18727 raft_consensus.cc:3037] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:44.970504 18727 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.970649 18727 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [CANDIDATE]: Term 1 election: Requested vote from peers 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607), 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817)
I20260807 20:10:44.970786 18526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f07573f580f4383a2b623eb3be01268" candidate_uuid: "79c19e31e8504c0594262c39a1592266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3235b0d1154209aeb79ee7a561ff81"
I20260807 20:10:44.970829 18382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f07573f580f4383a2b623eb3be01268" candidate_uuid: "79c19e31e8504c0594262c39a1592266" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8184af9eee254c2b99a8f04d39d56a70"
I20260807 20:10:44.970902 18382 raft_consensus.cc:3037] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:44.970894 18526 raft_consensus.cc:3037] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:10:44.971561 18382 raft_consensus.cc:2454] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c19e31e8504c0594262c39a1592266 in term 1.
I20260807 20:10:44.971673 18526 raft_consensus.cc:2454] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79c19e31e8504c0594262c39a1592266 in term 1.
I20260807 20:10:44.971732 18596 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c19e31e8504c0594262c39a1592266, 8184af9eee254c2b99a8f04d39d56a70; no voters:
I20260807 20:10:44.971818 18727 raft_consensus.cc:2781] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:10:44.971995 18727 raft_consensus.cc:686] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 1 LEADER]: Becoming Leader. State: Replica: 79c19e31e8504c0594262c39a1592266, State: Running, Role: LEADER
I20260807 20:10:44.972124 18727 consensus_queue.cc:237] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:44.973104 18234 catalog_manager.cc:5495] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79c19e31e8504c0594262c39a1592266 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "79c19e31e8504c0594262c39a1592266" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } health_report { overall_health: UNKNOWN } } }
W20260807 20:10:44.975840 18575 tablet.cc:2369] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:10:44.981400 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.979287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59170 (local address 127.3.15.131:45267)
0807 20:10:44.979351 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:44.979353 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:44.979362 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:10:44.979390 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:44.979391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:44.979406 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:44.979465 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:10:44.979574 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.980032 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.980694 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:44.980916 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:44.980969 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:44.981103 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:10:44.981207 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:10:44.981268 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:44.981309 (+ 41us) server_negotiation.cc:299] Negotiation successful
0807 20:10:44.981325 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:10:44.983158 18526 raft_consensus.cc:1261] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Refusing update from remote peer 79c19e31e8504c0594262c39a1592266: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:44.983158 18382 raft_consensus.cc:1261] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Refusing update from remote peer 79c19e31e8504c0594262c39a1592266: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:10:44.983575 18727 consensus_queue.cc:1035] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:44.983732 18730 consensus_queue.cc:1035] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:10:44.986027 18735 mvcc.cc:205] Tried to move back new op lower bound from 7316002590648483840 to 7316002590606667776. Current Snapshot: MvccSnapshot[applied={T|T < 7316002590648483840}]
I20260807 20:10:44.988200 18233 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:10:44.988301 18233 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57514:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:10:44.989198 18233 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Sending DeleteTablet for 3 replicas of tablet 4f07573f580f4383a2b623eb3be01268
I20260807 20:10:44.989523 18361 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f07573f580f4383a2b623eb3be01268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:39234
I20260807 20:10:44.989558 18643 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f07573f580f4383a2b623eb3be01268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59158
I20260807 20:10:44.989557 18505 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f07573f580f4383a2b623eb3be01268 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:10:44 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:45078
I20260807 20:10:44.989624 18438 tablet_replica.cc:330] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: stopping tablet replica
I20260807 20:10:44.989671 18438 raft_consensus.cc:2229] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:10:44.989728 18438 raft_consensus.cc:2258] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.989773 18744 tablet_replica.cc:330] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: stopping tablet replica
I20260807 20:10:44.989794 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18274
I20260807 20:10:44.989873 18744 raft_consensus.cc:2229] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:10:44.989971 18744 raft_consensus.cc:2258] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.990397 18743 tablet_replica.cc:330] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: stopping tablet replica
I20260807 20:10:44.990517 18743 raft_consensus.cc:2229] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:10:44.990633 18743 raft_consensus.cc:2258] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:10:44.991005 18744 ts_tablet_manager.cc:1905] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:44.993183 18744 ts_tablet_manager.cc:1918] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:44.993268 18744 log.cc:1196] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/4f07573f580f4383a2b623eb3be01268
I20260807 20:10:44.993568 18744 ts_tablet_manager.cc:1939] T 4f07573f580f4383a2b623eb3be01268 P 4f3235b0d1154209aeb79ee7a561ff81: Deleting consensus metadata
I20260807 20:10:44.993974 18217 catalog_manager.cc:4841] TS 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): tablet 4f07573f580f4383a2b623eb3be01268 (table test-table [id=fbe9658196ab457caa88e02b205f7229]) successfully deleted
I20260807 20:10:44.994463 18743 ts_tablet_manager.cc:1905] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:10:44.996501 18743 ts_tablet_manager.cc:1918] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:10:44.996583 18743 log.cc:1196] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/4f07573f580f4383a2b623eb3be01268
I20260807 20:10:44.996905 18743 ts_tablet_manager.cc:1939] T 4f07573f580f4383a2b623eb3be01268 P 79c19e31e8504c0594262c39a1592266: Deleting consensus metadata
I20260807 20:10:44.997329 18217 catalog_manager.cc:4841] TS 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): tablet 4f07573f580f4383a2b623eb3be01268 (table test-table [id=fbe9658196ab457caa88e02b205f7229]) successfully deleted
W20260807 20:10:44.999511 18220 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:40817 (error 108)
I20260807 20:10:44.999974 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18434
I20260807 20:10:44.999989 18718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:44.999864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:40817 (local address 127.0.0.1:39248)
0807 20:10:44.999912 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:44.999963 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:40817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260807 20:10:45.001441 18220 catalog_manager.cc:4584] TS 8184af9eee254c2b99a8f04d39d56a70 (127.3.15.129:40817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f07573f580f4383a2b623eb3be01268: Network error: Client connection negotiation failed: client connection to 127.3.15.129:40817: connect: Connection refused (error 111)
I20260807 20:10:45.009861 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18577
I20260807 20:10:45.016284 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18200
I20260807 20:10:45.022625 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:46207
--webserver_interface=127.3.15.190
--webserver_port=36353
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:46207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:45.106349 18746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:45.106596 18746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:45.106633 18746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:45.107764 18746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:10:45.107836 18746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:45.107860 18746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:45.107882 18746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:10:45.107913 18746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:10:45.109391 18746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:46207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:46207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=36353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:45.109670 18746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:45.109892 18746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:10:45.114984 18746 server_base.cc:1034] running on GCE node
W20260807 20:10:45.114923 18753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:45.114943 18752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:45.114919 18755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:45.115353 18746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:45.115649 18746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:45.116834 18746 hybrid_clock.cc:648] HybridClock initialized: now 1786133445116824 us; error 28 us; skew 500 ppm
Aug 07 20:10:45 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133445, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:45.119277 18746 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:10:45.119729 18746 webserver.cc:460] Webserver started at http://127.3.15.190:36353/ using document root <none> and password file <none>
I20260807 20:10:45.119908 18746 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:45.119948 18746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:45.121306 18746 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:45.122004 18762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.122225 18746 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:45.122313 18746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "e593bd2e00a14db0b624be17e624d48d"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "86203971a2c5c58d15c69a732c82aacb"
server_key_iv: "d929835e16643c5eb2a6ad8d6305eb3a"
server_key_version: "encryptionkey@0"
I20260807 20:10:45.122634 18746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:45.137696 18746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:45.139703 18746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:45.139822 18746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:45.144162 18746 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:46207
I20260807 20:10:45.144246 18815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:46207 every 8 connection(s)
I20260807 20:10:45.144652 18746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:10:45.146452 18816 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:10:45.147168 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap starting.
I20260807 20:10:45.148119 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18746
I20260807 20:10:45.148423 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:40817
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:46207
--webserver_port=40453
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:45.157070 18816 log.cc:824] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:45.159497 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:10:45.159840 18816 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d: Bootstrap complete.
I20260807 20:10:45.162391 18816 raft_consensus.cc:348] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:45.162650 18816 raft_consensus.cc:729] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e593bd2e00a14db0b624be17e624d48d, State: Initialized, Role: FOLLOWER
I20260807 20:10:45.162863 18816 consensus_queue.cc:260] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:45.162963 18816 raft_consensus.cc:388] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:10:45.163019 18816 raft_consensus.cc:482] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:10:45.163067 18816 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 2 FOLLOWER]: Advancing to term 3
I20260807 20:10:45.164116 18816 raft_consensus.cc:504] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:45.164229 18816 leader_election.cc:302] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e593bd2e00a14db0b624be17e624d48d; no voters:
I20260807 20:10:45.164448 18816 leader_election.cc:288] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 20:10:45.164484 18821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 3 FOLLOWER]: Leader election won for term 3
I20260807 20:10:45.164610 18821 raft_consensus.cc:686] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [term 3 LEADER]: Becoming Leader. State: Replica: e593bd2e00a14db0b624be17e624d48d, State: Running, Role: LEADER
I20260807 20:10:45.164726 18816 sys_catalog.cc:564] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:10:45.164713 18821 consensus_queue.cc:237] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } }
I20260807 20:10:45.165019 18821 sys_catalog.cc:455] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e593bd2e00a14db0b624be17e624d48d. Latest consensus state: current_term: 3 leader_uuid: "e593bd2e00a14db0b624be17e624d48d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } } }
I20260807 20:10:45.165102 18821 sys_catalog.cc:458] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:45.165026 18822 sys_catalog.cc:455] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e593bd2e00a14db0b624be17e624d48d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e593bd2e00a14db0b624be17e624d48d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 46207 } } }
I20260807 20:10:45.165340 18825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:10:45.165366 18822 sys_catalog.cc:458] T 00000000000000000000000000000000 P e593bd2e00a14db0b624be17e624d48d [sys.catalog]: This master's current role is: LEADER
I20260807 20:10:45.166283 18825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:10:45.166415 18825 catalog_manager.cc:1259] Loaded cluster ID: 32071f15414a4e8ebddfc26fa7fa8cfd
I20260807 20:10:45.166916 18825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 20:10:45.167219 18825 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0216b3e9ad google::LogMessage::Fail() at ??:0
@ 0x7f0216b4188a google::LogMessage::SendToLog() at ??:0
@ 0x7f0216b3e4de google::LogMessage::Flush() at ??:0
@ 0x7f0216b3fa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f021ea630f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f021ea6c8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0218a0c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0218a01954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f02198656db start_thread at ??:0
@ 0x7f021563c61f clone at ??:0
W20260807 20:10:45.251847 18819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:45.252060 18819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:45.252085 18819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:45.253320 18819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:45.253361 18819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:45.253388 18819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:10:45.254822 18819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:40817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=40453
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:45.255112 18819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:45.255362 18819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:45.261648 18854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:45.261704 18853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:45.261688 18856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:45.262280 18819 server_base.cc:1034] running on GCE node
I20260807 20:10:45.262542 18819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:45.262909 18819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:45.264119 18819 hybrid_clock.cc:648] HybridClock initialized: now 1786133445264100 us; error 40 us; skew 500 ppm
Aug 07 20:10:45 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133445, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:45.267632 18819 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:10:45.268224 18819 webserver.cc:460] Webserver started at http://127.3.15.129:40453/ using document root <none> and password file <none>
I20260807 20:10:45.268543 18819 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:45.268627 18819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:45.270589 18819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:45.271551 18863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.271767 18819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:45.271864 18819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "8184af9eee254c2b99a8f04d39d56a70"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "7f0c1f351d415401edfd78406d2e2fc0"
server_key_iv: "9a5bad36fb075568f43a6d878455571a"
server_key_version: "encryptionkey@0"
I20260807 20:10:45.272303 18819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:45.284053 18819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:45.286831 18819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:45.287004 18819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:45.287274 18819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:45.287797 18870 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:10:45.289996 18819 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:10:45.290078 18819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:10:45.290131 18819 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:10:45.290845 18819 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:10:45.290911 18819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 20:10:45.290979 18870 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap starting.
I20260807 20:10:45.299069 18819 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:40817
I20260807 20:10:45.299115 18982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:40817 every 8 connection(s)
I20260807 20:10:45.299554 18819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:10:45.302350 18870 log.cc:824] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Log is configured to *not* fsync() on all Append() calls
I20260807 20:10:45.304710 18870 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:10:45.305111 18870 tablet_bootstrap.cc:492] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Bootstrap complete.
I20260807 20:10:45.305287 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18819
I20260807 20:10:45.305584 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:33607
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:46207
--webserver_port=35341
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:10:45.306341 18870 ts_tablet_manager.cc:1397] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260807 20:10:45.307935 18870 raft_consensus.cc:348] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:45.308211 18870 raft_consensus.cc:729] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8184af9eee254c2b99a8f04d39d56a70, State: Initialized, Role: FOLLOWER
I20260807 20:10:45.308387 18870 consensus_queue.cc:260] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:45.308701 18870 ts_tablet_manager.cc:1428] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
W20260807 20:10:45.391817 18985 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:10:45.300325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:39545)
0807 20:10:45.300521 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.300567 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:45.301138 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:45.391560 (+ 90422us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
W20260807 20:10:45.392138 18983 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:46207 (0 consecutive failures): Network error: Failed to ping master at 127.3.15.190:46207: Client connection negotiation failed: client connection to 127.3.15.190:46207: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:10:45.392516 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.392388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:40767)
0807 20:10:45.392447 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.392486 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:10:45.392949 18985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.392858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:42889)
0807 20:10:45.392908 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.392934 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 20:10:45.393085 18983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:10:45.412302 18987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:45.412514 18987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:45.412540 18987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:45.414019 18987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:45.414079 18987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:45.414111 18987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:10:45.415768 18987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:33607
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=35341
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:45.416046 18987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:45.416286 18987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:45.421638 18997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:45.421645 18995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:45.421759 18987 server_base.cc:1034] running on GCE node
W20260807 20:10:45.421802 18994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:45.422035 18987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:45.422324 18987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:45.423501 18987 hybrid_clock.cc:648] HybridClock initialized: now 1786133445423484 us; error 26 us; skew 500 ppm
Aug 07 20:10:45 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133445, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:45.426081 18987 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:10:45.426570 18987 webserver.cc:460] Webserver started at http://127.3.15.130:35341/ using document root <none> and password file <none>
I20260807 20:10:45.426819 18987 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:45.426890 18987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:45.428247 18987 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:45.428947 19004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.429136 18987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:10:45.429241 18987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "4f3235b0d1154209aeb79ee7a561ff81"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "85ebf1abf284c97fefd0dba861324fdd"
server_key_iv: "a983b0a4788a6ab8f498093063166122"
server_key_version: "encryptionkey@0"
I20260807 20:10:45.429621 18987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:45.451082 18987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:45.453148 18987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:45.453263 18987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:45.453433 18987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:45.453745 18987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:45.453780 18987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.453802 18987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:45.453847 18987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.460071 18987 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:33607
I20260807 20:10:45.460145 19121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:33607 every 8 connection(s)
I20260807 20:10:45.460525 18987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:10:45.461627 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.461231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:54135)
0807 20:10:45.461398 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.461469 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":78,"threads_started":1}
W20260807 20:10:45.462021 19122 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:46207 (0 consecutive failures): Network error: Failed to ping master at 127.3.15.190:46207: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
I20260807 20:10:45.462349 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.462265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:50889)
0807 20:10:45.462299 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.462330 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 20:10:45.462651 19124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.462559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:45393)
0807 20:10:45.462607 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.462636 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 20:10:45.462913 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 18987
I20260807 20:10:45.463126 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:45267
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:46207
--webserver_port=41567
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:10:45.464722 19122 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:10:45.546419 19125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:10:45.546622 19125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:10:45.546644 19125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:10:45.547791 19125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:10:45.547835 19125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:10:45.547887 19125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:10:45.549361 19125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:45267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=41567
--tserver_master_addrs=127.3.15.190:46207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:10:45.549610 19125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:10:45.549860 19125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:10:45.554934 19130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:10:45.554908 19131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:45.555145 19125 server_base.cc:1034] running on GCE node
W20260807 20:10:45.554906 19133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:10:45.555471 19125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:10:45.555668 19125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:10:45.556846 19125 hybrid_clock.cc:648] HybridClock initialized: now 1786133445556830 us; error 29 us; skew 500 ppm
Aug 07 20:10:45 dist-test-slave-1x4g krb5kdc[17626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133445, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:10:45.559418 19125 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:10:45.559882 19125 webserver.cc:460] Webserver started at http://127.3.15.131:41567/ using document root <none> and password file <none>
I20260807 20:10:45.560129 19125 fs_manager.cc:359] Metadata directory not provided
I20260807 20:10:45.560199 19125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:10:45.561622 19125 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:10:45.562178 19140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.562362 19125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:10:45.562451 19125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "79c19e31e8504c0594262c39a1592266"
format_stamp: "Formatted at 2026-08-07 20:10:43 on dist-test-slave-1x4g"
server_key: "a5e5ae2076f5fded5b4e9e5247341150"
server_key_iv: "fb6b38ecaa410cdfeec7169d3c91fab0"
server_key_version: "encryptionkey@0"
I20260807 20:10:45.562829 19125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:10:45.577549 19125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:10:45.579875 19125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:10:45.580049 19125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:10:45.580296 19125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:10:45.580607 19125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:10:45.580662 19125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.580703 19125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:10:45.580730 19125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:10:45.586773 19125 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:45267
I20260807 20:10:45.586825 19259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:45267 every 8 connection(s)
I20260807 20:10:45.587168 19125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:10:45.588255 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.587842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:55727)
0807 20:10:45.588024 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.588111 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
W20260807 20:10:45.588696 19260 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:46207 (0 consecutive failures): Network error: Failed to ping master at 127.3.15.190:46207: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
I20260807 20:10:45.588934 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19125
I20260807 20:10:45.589102 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.588939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:57067)
0807 20:10:45.589031 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.589067 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 20:10:45.589700 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:45.589616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:54939)
0807 20:10:45.589662 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:45.589683 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 20:10:45.589834 19260 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:10:46.393688 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.393447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:48647)
0807 20:10:46.393576 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:46.393618 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260807 20:10:46.465497 19265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.465251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:34007)
0807 20:10:46.465397 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:46.465434 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260807 20:10:46.528621 19266 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:46.528825 19266 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:46.529222 19266 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
Aug 07 20:10:46 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133445, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.131@KRBTEST.COM
Aug 07 20:10:46 dist-test-slave-1x4g krb5kdc[17626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133445, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.130@KRBTEST.COM
I20260807 20:10:46.540438 19267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.529507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:55199 (local address 127.3.15.131:45267)
0807 20:10:46.529646 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:46.529659 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:46.529687 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:10:46.530170 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:46.530173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:46.530229 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:46.530410 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:46.531216 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.532213 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.533320 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.533720 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.533843 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:46.536210 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:46.536215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:46.536215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:46.536228 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:46.538423 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:46.538901 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:46.538904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:46.538904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:46.538996 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:46.539298 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:46.539299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:46.539299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:46.539734 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:46.539824 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:46.540195 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 20:10:46.540333 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":77,"threads_started":1}
I20260807 20:10:46.540455 19269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.529722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:56377 (local address 127.3.15.130:33607)
0807 20:10:46.529866 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 20:10:46.529875 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:10:46.529901 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:10:46.530327 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:10:46.530329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:10:46.530364 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:10:46.530505 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:10:46.531195 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.532158 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.533298 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.533711 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.533843 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:46.536808 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:10:46.536812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:10:46.536813 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:10:46.536825 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:10:46.538714 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:46.539057 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:46.539061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:46.539061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:46.539155 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:10:46.539462 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:10:46.539463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:10:46.539464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:10:46.539733 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:10:46.539824 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 20:10:46.540195 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 20:10:46.540337 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260807 20:10:46.540459 19264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.529389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:45267 (local address 127.3.15.129:55199)
0807 20:10:46.529442 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:46.529454 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:46.529513 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:46.530461 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:46.530464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:46.530851 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:46.531067 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:46.531073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.532333 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.532334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:46.533212 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:46.533214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.533816 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.533818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:46.533956 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:46.534606 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:46.534610 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:46.536083 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:46.538515 (+ 2432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:46.538522 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:46.538524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:46.538800 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:46.539096 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:46.539096 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:46.539097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:46.539217 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:46.539812 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:46.539821 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:46.539983 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 20:10:46.540115 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 20:10:46.540322 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":41}
I20260807 20:10:46.540853 19268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.529530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:33607 (local address 127.3.15.129:56377)
0807 20:10:46.529664 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:46.529675 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:10:46.529720 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:10:46.530514 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:10:46.530515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:10:46.530852 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:10:46.531067 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:46.531073 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.532260 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.532261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:46.533210 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:10:46.533213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:10:46.533817 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:10:46.533819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:10:46.533953 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:10:46.534540 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:10:46.534548 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:10:46.536449 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:10:46.538796 (+ 2347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:46.538797 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:46.538797 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:46.538981 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:46.539220 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:10:46.539221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:10:46.539221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:10:46.539296 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:10:46.539812 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:10:46.539821 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:10:46.539983 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 20:10:46.540122 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 20:10:46.540379 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":116,"thread_start_us":81,"threads_started":1}
W20260807 20:10:46.541467 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
W20260807 20:10:46.541787 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
I20260807 20:10:46.541847 18864 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8184af9eee254c2b99a8f04d39d56a70; no voters: 4f3235b0d1154209aeb79ee7a561ff81, 79c19e31e8504c0594262c39a1592266
I20260807 20:10:46.542021 19266 raft_consensus.cc:2726] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:10:46.590363 19270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:46.590147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:54089)
0807 20:10:46.590282 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:46.590324 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260807 20:10:47.394372 19271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:47.394151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:37663)
0807 20:10:47.394290 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:47.394325 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":64,"threads_started":1}
I20260807 20:10:47.466126 19272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:47.465923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:54723)
0807 20:10:47.466038 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:47.466071 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260807 20:10:47.591081 19273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:47.590882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:45199)
0807 20:10:47.591004 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:47.591040 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260807 20:10:48.395066 19274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:48.394848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:39117)
0807 20:10:48.394980 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:48.395018 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260807 20:10:48.466813 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:48.466605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:56947)
0807 20:10:48.466728 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:48.466765 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":64,"threads_started":1}
I20260807 20:10:48.591729 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:48.591525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:49619)
0807 20:10:48.591655 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:48.591687 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260807 20:10:48.737674 19277 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:48.737778 19277 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:48.737958 19277 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
W20260807 20:10:48.738404 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
W20260807 20:10:48.738479 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
I20260807 20:10:48.738495 18864 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8184af9eee254c2b99a8f04d39d56a70; no voters: 4f3235b0d1154209aeb79ee7a561ff81, 79c19e31e8504c0594262c39a1592266
I20260807 20:10:48.738561 19277 raft_consensus.cc:2726] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:10:49.395876 19278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:49.395627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:46953)
0807 20:10:49.395794 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:49.395830 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
I20260807 20:10:49.467512 19279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:49.467267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:60923)
0807 20:10:49.467415 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:49.467470 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260807 20:10:49.592357 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:49.592161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:47207)
0807 20:10:49.592280 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:49.592316 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":65,"threads_started":1}
I20260807 20:10:50.396674 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:50.396437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:42653)
0807 20:10:50.396574 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:50.396628 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260807 20:10:50.468232 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:50.468015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:36199)
0807 20:10:50.468145 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:50.468186 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260807 20:10:50.593125 19284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:50.592882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:57247)
0807 20:10:50.593040 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:50.593078 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260807 20:10:51.397456 19285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:51.397193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:41381)
0807 20:10:51.397357 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:51.397403 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260807 20:10:51.469002 19287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:51.468742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:56909)
0807 20:10:51.468883 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:51.468918 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260807 20:10:51.593757 19288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:51.593561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:57039)
0807 20:10:51.593681 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:51.593716 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260807 20:10:51.655779 19289 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:51.655871 19289 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:51.656075 19289 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
W20260807 20:10:51.656514 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
W20260807 20:10:51.656584 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
I20260807 20:10:51.656600 18864 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8184af9eee254c2b99a8f04d39d56a70; no voters: 4f3235b0d1154209aeb79ee7a561ff81, 79c19e31e8504c0594262c39a1592266
I20260807 20:10:51.656684 19289 raft_consensus.cc:2726] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:10:52.398259 19290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:52.398013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:37789)
0807 20:10:52.398164 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:52.398208 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260807 20:10:52.469650 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:52.469411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:59281)
0807 20:10:52.469571 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:52.469607 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260807 20:10:52.594409 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:52.594200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:45471)
0807 20:10:52.594329 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:52.594364 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260807 20:10:53.399037 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:53.398800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:50707)
0807 20:10:53.398953 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:53.398989 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260807 20:10:53.470314 19295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:53.470099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:50741)
0807 20:10:53.470228 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:53.470268 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":59,"threads_started":1}
I20260807 20:10:53.595185 19296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:53.594937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:55861)
0807 20:10:53.595102 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:53.595142 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260807 20:10:54.399827 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:54.399585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:33631)
0807 20:10:54.399732 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:54.399775 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260807 20:10:54.471024 19298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:54.470760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:44243)
0807 20:10:54.470905 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:54.470975 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":60,"threads_started":1}
I20260807 20:10:54.595834 19299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:54.595643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:46957)
0807 20:10:54.595760 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:54.595791 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260807 20:10:55.078018 19300 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:55.078169 19300 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:55.078384 19300 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
W20260807 20:10:55.078820 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
W20260807 20:10:55.078892 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
I20260807 20:10:55.078940 18864 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8184af9eee254c2b99a8f04d39d56a70; no voters: 4f3235b0d1154209aeb79ee7a561ff81, 79c19e31e8504c0594262c39a1592266
I20260807 20:10:55.079056 19300 raft_consensus.cc:2726] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:10:55.400591 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:55.400360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:40607)
0807 20:10:55.400500 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:55.400543 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":60,"threads_started":1}
I20260807 20:10:55.471777 19302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:55.471538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:54331)
0807 20:10:55.471693 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:55.471731 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":67,"threads_started":1}
I20260807 20:10:55.596513 19303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:55.596311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:58541)
0807 20:10:55.596434 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:55.596468 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260807 20:10:56.401423 19304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:56.401197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:36431)
0807 20:10:56.401334 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:56.401374 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260807 20:10:56.472520 19305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:56.472306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:59435)
0807 20:10:56.472439 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:56.472473 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260807 20:10:56.597301 19306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:56.597082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:57495)
0807 20:10:56.597221 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:56.597253 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260807 20:10:57.402174 19307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:57.401918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:59299)
0807 20:10:57.402077 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:57.402123 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":64,"threads_started":1}
I20260807 20:10:57.473275 19308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:57.473057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:56829)
0807 20:10:57.473192 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:57.473228 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260807 20:10:57.598001 19309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:57.597770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:47365)
0807 20:10:57.597915 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:57.597955 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260807 20:10:58.402958 19310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:58.402719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:35107)
0807 20:10:58.402871 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:58.402910 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":68,"threads_started":1}
I20260807 20:10:58.473955 19311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:58.473746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:51359)
0807 20:10:58.473871 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:58.473907 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260807 20:10:58.598667 19312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:58.598452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:39687)
0807 20:10:58.598576 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:58.598621 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":66,"threads_started":1}
I20260807 20:10:59.403788 19313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:59.403528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:56831)
0807 20:10:59.403682 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:59.403734 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260807 20:10:59.474653 19314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:59.474452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:33643)
0807 20:10:59.474575 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:59.474609 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":64,"threads_started":1}
I20260807 20:10:59.599355 19315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:10:59.599120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:52459)
0807 20:10:59.599269 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:10:59.599305 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260807 20:10:59.640403 19316 raft_consensus.cc:482] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:10:59.640497 19316 raft_consensus.cc:504] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c19e31e8504c0594262c39a1592266" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45267 } } peers { permanent_uuid: "4f3235b0d1154209aeb79ee7a561ff81" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 33607 } } peers { permanent_uuid: "8184af9eee254c2b99a8f04d39d56a70" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 40817 } }
I20260807 20:10:59.640681 19316 leader_election.cc:288] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267), 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607)
W20260807 20:10:59.641129 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79c19e31e8504c0594262c39a1592266 (127.3.15.131:45267): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
W20260807 20:10:59.641233 18864 leader_election.cc:341] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4f3235b0d1154209aeb79ee7a561ff81 (127.3.15.130:33607): Not found: Tablet not found: 4f07573f580f4383a2b623eb3be01268
I20260807 20:10:59.641268 18864 leader_election.cc:302] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8184af9eee254c2b99a8f04d39d56a70; no voters: 4f3235b0d1154209aeb79ee7a561ff81, 79c19e31e8504c0594262c39a1592266
I20260807 20:10:59.641369 19316 raft_consensus.cc:2726] T 4f07573f580f4383a2b623eb3be01268 P 8184af9eee254c2b99a8f04d39d56a70 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:11:00.404731 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:00.404465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.129:43983)
0807 20:11:00.404632 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:00.404681 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":70,"threads_started":1}
I20260807 20:11:00.475500 19318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:00.475169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.130:55683)
0807 20:11:00.475388 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:00.475454 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260807 20:11:00.598613 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18819
I20260807 20:11:00.600492 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:00.600259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:46207 (local address 127.3.15.131:58287)
0807 20:11:00.600393 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:00.600431 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:46207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":57,"threads_started":1}
I20260807 20:11:00.606086 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 18987
I20260807 20:11:00.614347 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19125
2026-08-07T20:11:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18590 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:11:00 dist-test-slave-1x4g krb5kdc[19322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:11:00 dist-test-slave-1x4g krb5kdc[19322](info): set up 2 sockets
Aug 07 20:11:00 dist-test-slave-1x4g krb5kdc[19322](info): commencing operation
krb5kdc: starting...
W20260807 20:11:01.706748 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:11:01 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:11:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:11:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:11:01.834148 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45179
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:45179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:11:01.918447 19336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:01.918713 19336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:01.918753 19336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:01.919924 19336 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 20:11:01.919996 19336 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 20:11:01.920017 19336 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 20:11:01.920034 19336 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:11:01.920061 19336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:01.920105 19336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:01.920135 19336 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:11:01.920164 19336 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:11:01.921736 19336 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:45179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:01.922034 19336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:01.922291 19336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:01.927480 19342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:01.927484 19344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:01.927517 19336 server_base.cc:1034] running on GCE node
W20260807 20:11:01.927484 19341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:01.927948 19336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:01.928280 19336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:01.929428 19336 hybrid_clock.cc:648] HybridClock initialized: now 1786133461929420 us; error 38 us; skew 500 ppm
Aug 07 20:11:01 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133461, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:01.932020 19336 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:11:01.932494 19336 webserver.cc:460] Webserver started at http://127.3.15.190:40461/ using document root <none> and password file <none>
I20260807 20:11:01.932699 19336 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:01.932741 19336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:01.932893 19336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:11:01.933951 19336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "25f3e79a9a054d6aad0b444522328245"
format_stamp: "Formatted at 2026-08-07 20:11:01 on dist-test-slave-1x4g"
server_key: "301da9b6b228c114a8dd3eba948e986b"
server_key_iv: "d1153c2ea044701b0a1acfac76b3f6ea"
server_key_version: "encryptionkey@0"
I20260807 20:11:01.934266 19336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "25f3e79a9a054d6aad0b444522328245"
format_stamp: "Formatted at 2026-08-07 20:11:01 on dist-test-slave-1x4g"
server_key: "301da9b6b228c114a8dd3eba948e986b"
server_key_iv: "d1153c2ea044701b0a1acfac76b3f6ea"
server_key_version: "encryptionkey@0"
I20260807 20:11:01.935559 19336 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:11:01.936390 19352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:01.936643 19336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:01.936743 19336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "25f3e79a9a054d6aad0b444522328245"
format_stamp: "Formatted at 2026-08-07 20:11:01 on dist-test-slave-1x4g"
server_key: "301da9b6b228c114a8dd3eba948e986b"
server_key_iv: "d1153c2ea044701b0a1acfac76b3f6ea"
server_key_version: "encryptionkey@0"
I20260807 20:11:01.936846 19336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:01.946025 19336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:01.947950 19336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:01.948103 19336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:01.952519 19336 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:45179
I20260807 20:11:01.952554 19406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:45179 every 8 connection(s)
I20260807 20:11:01.952943 19336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:11:01.953923 19407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:01.956341 19407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap starting.
I20260807 20:11:01.957021 19407 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:01.957340 19407 log.cc:824] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:01.958093 19407 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: No bootstrap required, opened a new log
I20260807 20:11:01.959743 19407 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:01.959894 19407 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:01.959973 19407 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f3e79a9a054d6aad0b444522328245, State: Initialized, Role: FOLLOWER
I20260807 20:11:01.960129 19407 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:01.960187 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19336
I20260807 20:11:01.960219 19407 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:11:01.960259 19407 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:11:01.960312 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:11:01.960330 19407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:01.960518 3134 external_mini_cluster.cc:1428] Setting key 1a37839c9802eb3e82f71490bea4b241
I20260807 20:11:01.960981 19407 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:01.961113 19407 leader_election.cc:302] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25f3e79a9a054d6aad0b444522328245; no voters:
I20260807 20:11:01.961310 19407 leader_election.cc:288] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:11:01.961606 19407 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:11:01.961637 19412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:11:01.962762 19412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 LEADER]: Becoming Leader. State: Replica: 25f3e79a9a054d6aad0b444522328245, State: Running, Role: LEADER
I20260807 20:11:01.962891 19412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:01.963405 19421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25f3e79a9a054d6aad0b444522328245. Latest consensus state: current_term: 1 leader_uuid: "25f3e79a9a054d6aad0b444522328245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } } }
I20260807 20:11:01.963521 19421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: This master's current role is: LEADER
I20260807 20:11:01.963526 19413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25f3e79a9a054d6aad0b444522328245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } } }
I20260807 20:11:01.963967 19413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: This master's current role is: LEADER
I20260807 20:11:01.964012 19427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:11:01.964567 19427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:11:01.966291 19427 catalog_manager.cc:1347] Generated new cluster ID: b2f7c1dae1a24300a2d897a33c8c0f75
I20260807 20:11:01.966346 19427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:11:01 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:01.972944 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:01.961130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.3.15.190:45179)
0807 20:11:01.961346 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:01.961358 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:01.961393 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:11:01.961535 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:01.961542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:01.961613 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:01.961760 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:01.962502 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:01.964079 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:01.965232 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:01.965858 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:01.965975 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:01.968887 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:01.968892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:01.968892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:01.968906 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:01.971160 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:01.971553 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:01.971557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:01.971557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:01.971690 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:01.971915 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:01.971916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:01.971916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:01.972270 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:01.972425 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:01.972540 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 20:11:01.972691 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":110,"threads_started":1}
I20260807 20:11:01.973522 19427 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:11:01.974052 19427 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:11:01.977339 19427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Generated new TSK 0
I20260807 20:11:01.977512 19427 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:11:02.007915 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:45179
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:11:02.093968 19433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:02.094172 19433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:02.094195 19433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:02.095367 19433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:02.095408 19433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:02.095438 19433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:11:02.097078 19433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19433
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:02.097340 19433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:02.097587 19433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:02.102909 19440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:02.102916 19442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:02.103062 19433 server_base.cc:1034] running on GCE node
W20260807 20:11:02.102921 19439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:02.103401 19433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:02.103680 19433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:02.104841 19433 hybrid_clock.cc:648] HybridClock initialized: now 1786133462104825 us; error 44 us; skew 500 ppm
Aug 07 20:11:02 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:02.107307 19433 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:11:02.107738 19433 webserver.cc:460] Webserver started at http://127.3.15.129:41035/ using document root <none> and password file <none>
I20260807 20:11:02.107980 19433 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:02.108047 19433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:02.108202 19433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:11:02.109323 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "94ad79cfc59148399d7c0fabd4c25d39"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "46b3c78788b4db27d44d531ceb01de25"
server_key_iv: "35369cad9beb91b7efb91c41aa8d34be"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.109702 19433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "94ad79cfc59148399d7c0fabd4c25d39"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "46b3c78788b4db27d44d531ceb01de25"
server_key_iv: "35369cad9beb91b7efb91c41aa8d34be"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.111032 19433 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:11:02.111783 19449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.111994 19433 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:02.112097 19433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "94ad79cfc59148399d7c0fabd4c25d39"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "46b3c78788b4db27d44d531ceb01de25"
server_key_iv: "35369cad9beb91b7efb91c41aa8d34be"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.112174 19433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:02.128167 19433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:02.130156 19433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:02.130324 19433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:02.130563 19433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:02.130894 19433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:02.130949 19433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.130995 19433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:02.131018 19433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.137089 19433 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:37155
I20260807 20:11:02.137136 19564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:37155 every 8 connection(s)
I20260807 20:11:02.137501 19433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
Aug 07 20:11:02 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:02.143971 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19433
I20260807 20:11:02.144088 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:11:02.144294 3134 external_mini_cluster.cc:1428] Setting key 6c99edada29ef10dfe677936c12bf40f
I20260807 20:11:02.147722 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.138342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:32775 (local address 127.3.15.190:45179)
0807 20:11:02.138425 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.138428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.138575 (+ 147us) server_negotiation.cc:407] Connection header received
0807 20:11:02.139060 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.139061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.139076 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.139143 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:02.140193 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.140826 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.141818 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.142014 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.142066 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.144325 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:02.144327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:02.144327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:02.144338 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:02.146110 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.146572 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.146574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.146574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.146640 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.146931 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.146933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.146934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.147146 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:02.147213 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.147572 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.147652 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 20:11:02.148026 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.138249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:32775)
0807 20:11:02.138421 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.138465 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.138974 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.139229 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.139232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.139881 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:02.140109 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.140114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.140918 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.140919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.141744 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.141746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.142087 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.142088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.142191 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.142737 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:02.142746 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:02.144230 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:02.146221 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:02.146229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:02.146231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:02.146467 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:02.146716 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:02.146717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:02.146718 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:02.146829 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:02.147226 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:02.147234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:02.147414 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.147567 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.147753 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260807 20:11:02.148583 19565 heartbeater.cc:347] Connected to a master server at 127.3.15.190:45179
I20260807 20:11:02.148694 19565 heartbeater.cc:464] Registering TS with master...
I20260807 20:11:02.148918 19565 heartbeater.cc:511] Master 127.3.15.190:45179 requested a full tablet report, sending...
I20260807 20:11:02.149515 19369 ts_manager.cc:194] Registered new tserver with Master: 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155)
I20260807 20:11:02.150202 19369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:32775
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:11:02.173741 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:45179
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:11:02.258889 19570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:02.259124 19570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:02.259150 19570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:02.260353 19570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:02.260398 19570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:02.260454 19570 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:11:02.262058 19570 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:02.262305 19570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:02.262571 19570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:02.267719 19575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:02.267741 19576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:02.267865 19570 server_base.cc:1034] running on GCE node
W20260807 20:11:02.267765 19578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:02.268121 19570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:02.268318 19570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:02.269454 19570 hybrid_clock.cc:648] HybridClock initialized: now 1786133462269412 us; error 70 us; skew 500 ppm
Aug 07 20:11:02 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:02.272027 19570 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:11:02.272482 19570 webserver.cc:460] Webserver started at http://127.3.15.130:37399/ using document root <none> and password file <none>
I20260807 20:11:02.272670 19570 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:02.272712 19570 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:02.272861 19570 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:11:02.273921 19570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "1f47d3a4984a4ae78b77fecc6abdbb18"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "1ca1a921bbf4cba5a431e6e2cb9dcaee"
server_key_iv: "5e00aa14d140d7cdb89fc3b5619d46f0"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.274242 19570 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "1f47d3a4984a4ae78b77fecc6abdbb18"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "1ca1a921bbf4cba5a431e6e2cb9dcaee"
server_key_iv: "5e00aa14d140d7cdb89fc3b5619d46f0"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.275484 19570 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:11:02.276166 19585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.276338 19570 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:11:02.276445 19570 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "1f47d3a4984a4ae78b77fecc6abdbb18"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "1ca1a921bbf4cba5a431e6e2cb9dcaee"
server_key_iv: "5e00aa14d140d7cdb89fc3b5619d46f0"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.276535 19570 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:02.284415 19570 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:02.286743 19570 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:02.286903 19570 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:02.287127 19570 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:02.287436 19570 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:02.287492 19570 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.287539 19570 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:02.287567 19570 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.293404 19570 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:42485
I20260807 20:11:02.293453 19707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:42485 every 8 connection(s)
I20260807 20:11:02.293781 19570 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:11:02.299746 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19570
I20260807 20:11:02.299852 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:11:02.300088 3134 external_mini_cluster.cc:1428] Setting key 368b830b91dee18f8e1bccc8e1b7e0c4
Aug 07 20:11:02 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:02.304255 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.294572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:37305 (local address 127.3.15.190:45179)
0807 20:11:02.294644 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.294647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.294930 (+ 283us) server_negotiation.cc:407] Connection header received
0807 20:11:02.295622 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.295624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.295638 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.295699 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:02.296943 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.297607 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.298573 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.298765 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.298817 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.301399 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:02.301400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:02.301400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:02.301407 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:02.302805 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.303226 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.303227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.303228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.303296 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.303585 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.303586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.303586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.303706 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:02.303784 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.304065 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.304163 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:11:02.304461 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.294532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:37305)
0807 20:11:02.294759 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.294808 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.295499 (+ 691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.295798 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.295802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.296559 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:02.296831 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.296839 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.297696 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.297697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.298497 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.298499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.298839 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.298840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.298946 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.299407 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:02.299415 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:02.301308 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:02.302897 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:02.302904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:02.302906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:02.303144 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:02.303365 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:02.303366 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:02.303367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:02.303486 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:02.303774 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:02.303782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:02.303927 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.304041 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.304202 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260807 20:11:02.305063 19708 heartbeater.cc:347] Connected to a master server at 127.3.15.190:45179
I20260807 20:11:02.305184 19708 heartbeater.cc:464] Registering TS with master...
I20260807 20:11:02.305398 19708 heartbeater.cc:511] Master 127.3.15.190:45179 requested a full tablet report, sending...
I20260807 20:11:02.305840 19369 ts_manager.cc:194] Registered new tserver with Master: 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485)
I20260807 20:11:02.306226 19369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:37305
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:11:02.327297 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:45179
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:11:02.412025 19714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:02.412252 19714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:02.412278 19714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:02.413528 19714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:02.413576 19714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:02.413604 19714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:11:02.415037 19714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:02.415349 19714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:02.415567 19714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:02.420672 19720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:02.420672 19721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:02.420696 19723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:02.420951 19714 server_base.cc:1034] running on GCE node
I20260807 20:11:02.421222 19714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:02.421502 19714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:02.422616 19714 hybrid_clock.cc:648] HybridClock initialized: now 1786133462422602 us; error 31 us; skew 500 ppm
Aug 07 20:11:02 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:02.425324 19714 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:11:02.425778 19714 webserver.cc:460] Webserver started at http://127.3.15.131:38875/ using document root <none> and password file <none>
I20260807 20:11:02.426023 19714 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:02.426093 19714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:02.426245 19714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:11:02.427340 19714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "06cdb630a1174bdd951fdec8d77d1360"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "9626cf1d065cfaf8d74388758242615b"
server_key_iv: "b323563e01c71785a876dfff62382d9d"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.427722 19714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "06cdb630a1174bdd951fdec8d77d1360"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "9626cf1d065cfaf8d74388758242615b"
server_key_iv: "b323563e01c71785a876dfff62382d9d"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.429157 19714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:11:02.429949 19731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.430127 19714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:11:02.430233 19714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "06cdb630a1174bdd951fdec8d77d1360"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "9626cf1d065cfaf8d74388758242615b"
server_key_iv: "b323563e01c71785a876dfff62382d9d"
server_key_version: "encryptionkey@0"
I20260807 20:11:02.430315 19714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:02.442716 19714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:02.444497 19714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:02.444655 19714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:02.444952 19714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:02.445279 19714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:02.445333 19714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.445385 19714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:02.445413 19714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:02.451083 19714 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:34547
I20260807 20:11:02.451193 19851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:34547 every 8 connection(s)
I20260807 20:11:02.451556 19714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:11:02.453189 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19714
I20260807 20:11:02.453306 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:11:02.453501 3134 external_mini_cluster.cc:1428] Setting key bc0ce5372c76d0d2fd69a25fa8684b71
Aug 07 20:11:02 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133462, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:02.461192 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.452309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:33843 (local address 127.3.15.190:45179)
0807 20:11:02.452415 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.452417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.452594 (+ 177us) server_negotiation.cc:407] Connection header received
0807 20:11:02.453306 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.453308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.453321 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.453411 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:02.454750 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.455241 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.456247 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.456424 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.456474 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.458601 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:02.458603 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:02.458603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:02.458610 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:02.459863 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.460230 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.460231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.460231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.460284 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.460531 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.460531 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.460532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.460643 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:02.460693 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.461045 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.461128 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 20:11:02.461424 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.452282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:33843)
0807 20:11:02.452463 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.452499 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.453163 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.453554 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.453559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.454422 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:02.454670 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.454675 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.455329 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.455329 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.456170 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.456172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.456496 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.456497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.456602 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.457088 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:02.457097 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:02.458509 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:02.459939 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:02.459945 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:02.459946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:02.460163 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:02.460354 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:02.460355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:02.460355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:02.460474 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:02.460700 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:02.460705 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:02.460884 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.461008 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.461153 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260807 20:11:02.461951 19852 heartbeater.cc:347] Connected to a master server at 127.3.15.190:45179
I20260807 20:11:02.462046 19852 heartbeater.cc:464] Registering TS with master...
I20260807 20:11:02.462268 19852 heartbeater.cc:511] Master 127.3.15.190:45179 requested a full tablet report, sending...
I20260807 20:11:02.462628 19369 ts_manager.cc:194] Registered new tserver with Master: 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
I20260807 20:11:02.463088 19369 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:33843
I20260807 20:11:02.465210 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:11:02.472743 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.466618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.3.15.190:45179)
0807 20:11:02.466683 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.466684 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.466800 (+ 116us) server_negotiation.cc:407] Connection header received
0807 20:11:02.466844 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.466845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.466854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.466929 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:02.467454 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.467943 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.468647 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.468908 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.468959 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.470308 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:02.470309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:02.470309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:02.470316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:02.471745 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.472058 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.472059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.472059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.472117 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:02.472288 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:02.472288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:02.472289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:02.472420 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:02.472498 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.472603 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.472659 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:11:02.474212 19369 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:11:02.475766 19369 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:11:02.482298 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.478105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:34547 (local address 127.0.0.1:58806)
0807 20:11:02.478464 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.478473 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.478519 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.478776 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.478777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.478782 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:02.478846 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.478849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.480255 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.480256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.481120 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.481123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.481973 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.481974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.482054 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.482069 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.482154 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.482218 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":7,"thread_start_us":51,"threads_started":1}
I20260807 20:11:02.482326 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.478292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58806 (local address 127.3.15.131:34547)
0807 20:11:02.478479 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.478485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.478520 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:11:02.478588 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.478590 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.478609 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.478697 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:02.478953 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.480120 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.481210 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.481872 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.481961 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.482026 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.482183 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.482226 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260807 20:11:02.482522 19861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.478105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:37155 (local address 127.0.0.1:38614)
0807 20:11:02.478346 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.478355 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.478398 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.478775 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.478775 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.478778 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:02.478860 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.478862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.479905 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.479906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.481485 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.481487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.482307 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.482308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.482371 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.482374 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.482430 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.482466 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":22,"thread_start_us":54,"threads_started":1}
I20260807 20:11:02.482566 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.478318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.3.15.129:37155)
0807 20:11:02.478534 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.478539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.478550 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:11:02.478599 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.478601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.478616 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.478688 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:02.478947 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.479801 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.481568 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.482158 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.482218 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.482273 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.482447 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.482484 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":120,"threads_started":1}
I20260807 20:11:02.482996 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.478063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42485 (local address 127.0.0.1:43426)
0807 20:11:02.478220 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.478261 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.478325 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.478764 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.478768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.478771 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:02.478846 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.478850 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.480227 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.480229 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.481375 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.481378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.482311 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.482313 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.482844 (+ 531us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.482849 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.482889 (+ 40us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.482929 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":5,"thread_start_us":53,"threads_started":1}
I20260807 20:11:02.483116 19781 tablet_service.cc:1467] Processing CreateTablet for tablet b93409b382d84cbb947c8348b0b1f898 (DEFAULT_TABLE table=test-table [id=cb9b0caf1f414d26a720dd4f827a2962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:11:02.483412 19863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.478170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43426 (local address 127.3.15.130:42485)
0807 20:11:02.478578 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.478582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.478591 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:11:02.478630 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.478633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.478649 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.478932 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:02.479016 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.480092 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.481466 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.482208 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.483198 (+ 990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.483261 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.483289 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.483328 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":89,"threads_started":1}
I20260807 20:11:02.483496 19781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b93409b382d84cbb947c8348b0b1f898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:02.484095 19635 tablet_service.cc:1467] Processing CreateTablet for tablet b93409b382d84cbb947c8348b0b1f898 (DEFAULT_TABLE table=test-table [id=cb9b0caf1f414d26a720dd4f827a2962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:11:02.484441 19635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b93409b382d84cbb947c8348b0b1f898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:02.485133 19498 tablet_service.cc:1467] Processing CreateTablet for tablet b93409b382d84cbb947c8348b0b1f898 (DEFAULT_TABLE table=test-table [id=cb9b0caf1f414d26a720dd4f827a2962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:11:02.485436 19498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b93409b382d84cbb947c8348b0b1f898. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:02.487082 19867 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Bootstrap starting.
I20260807 20:11:02.487710 19868 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Bootstrap starting.
I20260807 20:11:02.487897 19867 tablet_bootstrap.cc:654] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:02.488340 19868 tablet_bootstrap.cc:654] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:02.488579 19869 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap starting.
I20260807 20:11:02.488649 19867 log.cc:824] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:02.489035 19868 log.cc:824] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:02.489382 19869 tablet_bootstrap.cc:654] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:02.489639 19867 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: No bootstrap required, opened a new log
I20260807 20:11:02.489698 19867 ts_tablet_manager.cc:1397] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:11:02.489975 19869 log.cc:824] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:02.490644 19869 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: No bootstrap required, opened a new log
I20260807 20:11:02.490728 19869 ts_tablet_manager.cc:1397] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:11:02.491308 19867 raft_consensus.cc:348] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.491447 19867 raft_consensus.cc:374] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:02.491470 19867 raft_consensus.cc:729] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06cdb630a1174bdd951fdec8d77d1360, State: Initialized, Role: FOLLOWER
I20260807 20:11:02.491601 19867 consensus_queue.cc:260] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.491919 19867 ts_tablet_manager.cc:1428] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:11:02.491941 19852 heartbeater.cc:503] Master 127.3.15.190:45179 was elected leader, sending a full tablet report...
I20260807 20:11:02.492484 19869 raft_consensus.cc:348] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.492659 19869 raft_consensus.cc:374] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:02.492638 19868 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: No bootstrap required, opened a new log
I20260807 20:11:02.492712 19869 raft_consensus.cc:729] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ad79cfc59148399d7c0fabd4c25d39, State: Initialized, Role: FOLLOWER
I20260807 20:11:02.492769 19868 ts_tablet_manager.cc:1397] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260807 20:11:02.492897 19869 consensus_queue.cc:260] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.493180 19869 ts_tablet_manager.cc:1428] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 20:11:02.493302 19565 heartbeater.cc:503] Master 127.3.15.190:45179 was elected leader, sending a full tablet report...
I20260807 20:11:02.494917 19868 raft_consensus.cc:348] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.495057 19868 raft_consensus.cc:374] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:02.495079 19868 raft_consensus.cc:729] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f47d3a4984a4ae78b77fecc6abdbb18, State: Initialized, Role: FOLLOWER
I20260807 20:11:02.495187 19868 consensus_queue.cc:260] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.495419 19868 ts_tablet_manager.cc:1428] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:11:02.495471 19708 heartbeater.cc:503] Master 127.3.15.190:45179 was elected leader, sending a full tablet report...
I20260807 20:11:02.517957 19873 raft_consensus.cc:482] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:02.518080 19873 raft_consensus.cc:504] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.518514 19873 leader_election.cc:288] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155)
I20260807 20:11:02.522476 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.518819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:39099 (local address 127.3.15.129:37155)
0807 20:11:02.518865 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.518866 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.519005 (+ 139us) server_negotiation.cc:407] Connection header received
0807 20:11:02.519045 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.519046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.519056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.519100 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:02.519257 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.519992 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.521489 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.522101 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.522180 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.522189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.522361 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.522403 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:11:02.522485 19863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.518775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:60635 (local address 127.3.15.130:42485)
0807 20:11:02.518832 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.518834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.518844 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:11:02.518878 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.518879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.518898 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.518949 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:02.519181 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.519958 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.521489 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.522097 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.522180 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.522189 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.522361 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.522403 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:11:02.522492 19854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.518687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42485 (local address 127.3.15.131:60635)
0807 20:11:02.518748 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.518763 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.518814 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.518979 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.518980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.518982 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:02.519065 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.519069 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.520066 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.520066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.521387 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.521391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.522188 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.522189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.522275 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.522282 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.522340 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.522394 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":44}
I20260807 20:11:02.522502 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.518718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:37155 (local address 127.3.15.131:39099)
0807 20:11:02.518908 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.518924 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:02.518992 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:02.519120 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:02.519121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:02.519122 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:02.519189 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.519190 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.520066 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.520066 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.521387 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:02.521391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.522188 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.522189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:02.522275 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.522282 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 20:11:02.522340 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:11:02.522394 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":70,"thread_start_us":114,"threads_started":1}
I20260807 20:11:02.523085 19658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93409b382d84cbb947c8348b0b1f898" candidate_uuid: "06cdb630a1174bdd951fdec8d77d1360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" is_pre_election: true
I20260807 20:11:02.523180 19518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93409b382d84cbb947c8348b0b1f898" candidate_uuid: "06cdb630a1174bdd951fdec8d77d1360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ad79cfc59148399d7c0fabd4c25d39" is_pre_election: true
I20260807 20:11:02.523262 19658 raft_consensus.cc:2454] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06cdb630a1174bdd951fdec8d77d1360 in term 0.
I20260807 20:11:02.523357 19518 raft_consensus.cc:2454] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 06cdb630a1174bdd951fdec8d77d1360 in term 0.
I20260807 20:11:02.523589 19732 leader_election.cc:302] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18; no voters:
I20260807 20:11:02.523782 19873 raft_consensus.cc:2781] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:11:02.523850 19873 raft_consensus.cc:482] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:11:02.523867 19873 raft_consensus.cc:3037] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:02.524621 19873 raft_consensus.cc:504] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.524794 19873 leader_election.cc:288] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [CANDIDATE]: Term 1 election: Requested vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155)
I20260807 20:11:02.524959 19518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93409b382d84cbb947c8348b0b1f898" candidate_uuid: "06cdb630a1174bdd951fdec8d77d1360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ad79cfc59148399d7c0fabd4c25d39"
I20260807 20:11:02.525012 19658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b93409b382d84cbb947c8348b0b1f898" candidate_uuid: "06cdb630a1174bdd951fdec8d77d1360" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18"
I20260807 20:11:02.525045 19518 raft_consensus.cc:3037] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:02.525070 19658 raft_consensus.cc:3037] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:02.525797 19518 raft_consensus.cc:2454] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06cdb630a1174bdd951fdec8d77d1360 in term 1.
I20260807 20:11:02.525875 19658 raft_consensus.cc:2454] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 06cdb630a1174bdd951fdec8d77d1360 in term 1.
I20260807 20:11:02.526032 19733 leader_election.cc:302] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cdb630a1174bdd951fdec8d77d1360, 94ad79cfc59148399d7c0fabd4c25d39; no voters:
I20260807 20:11:02.526165 19873 raft_consensus.cc:2781] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:11:02.526356 19873 raft_consensus.cc:686] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 LEADER]: Becoming Leader. State: Replica: 06cdb630a1174bdd951fdec8d77d1360, State: Running, Role: LEADER
I20260807 20:11:02.526469 19873 consensus_queue.cc:237] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:02.527395 19369 catalog_manager.cc:5495] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 reported cstate change: term changed from 0 to 1, leader changed from <none> to 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "06cdb630a1174bdd951fdec8d77d1360" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } health_report { overall_health: HEALTHY } } }
I20260807 20:11:02.539645 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.537487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.3.15.131:34547)
0807 20:11:02.537545 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:02.537547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:02.537554 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:11:02.537581 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:02.537582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:02.537598 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:02.537656 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:11:02.537779 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.538233 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.538919 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:02.539103 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:02.539152 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:02.539322 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:11:02.539413 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:11:02.539475 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:02.539512 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:11:02.539529 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:11:02.541489 19518 raft_consensus.cc:1261] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Refusing update from remote peer 06cdb630a1174bdd951fdec8d77d1360: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:11:02.541522 19658 raft_consensus.cc:1261] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Refusing update from remote peer 06cdb630a1174bdd951fdec8d77d1360: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:11:02.541802 19873 consensus_queue.cc:1035] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:11:02.541945 19877 consensus_queue.cc:1035] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:11:02.544327 19709 tablet.cc:2369] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:11:02.544644 19880 mvcc.cc:205] Tried to move back new op lower bound from 7316002662567383040 to 7316002662509301760. Current Snapshot: MvccSnapshot[applied={T|T < 7316002662567383040}]
I20260807 20:11:02.545853 19881 mvcc.cc:205] Tried to move back new op lower bound from 7316002662567383040 to 7316002662509301760. Current Snapshot: MvccSnapshot[applied={T|T < 7316002662567383040}]
I20260807 20:11:02.547201 19369 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:11:02.547309 19369 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:11:02.548113 19369 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Sending DeleteTablet for 3 replicas of tablet b93409b382d84cbb947c8348b0b1f898
I20260807 20:11:02.548461 19498 tablet_service.cc:1514] Processing DeleteTablet for tablet b93409b382d84cbb947c8348b0b1f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:02 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:38614
I20260807 20:11:02.548460 19635 tablet_service.cc:1514] Processing DeleteTablet for tablet b93409b382d84cbb947c8348b0b1f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:02 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:43426
I20260807 20:11:02.548504 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19433
I20260807 20:11:02.551141 19781 tablet_service.cc:1514] Processing DeleteTablet for tablet b93409b382d84cbb947c8348b0b1f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:02 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:58806
I20260807 20:11:02.551293 19892 tablet_replica.cc:330] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: stopping tablet replica
I20260807 20:11:02.551384 19892 raft_consensus.cc:2229] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:11:02.551472 19892 raft_consensus.cc:2258] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:11:02.551872 19892 ts_tablet_manager.cc:1905] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:11:02.552906 19893 tablet_replica.cc:330] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: stopping tablet replica
I20260807 20:11:02.552999 19893 raft_consensus.cc:2229] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:11:02.553092 19893 raft_consensus.cc:2258] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:11:02.553419 19892 ts_tablet_manager.cc:1918] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:11:02.553449 19893 ts_tablet_manager.cc:1905] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:11:02.553486 19892 log.cc:1196] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/b93409b382d84cbb947c8348b0b1f898
I20260807 20:11:02.553786 19892 ts_tablet_manager.cc:1939] T b93409b382d84cbb947c8348b0b1f898 P 1f47d3a4984a4ae78b77fecc6abdbb18: Deleting consensus metadata
I20260807 20:11:02.554207 19353 catalog_manager.cc:4841] TS 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): tablet b93409b382d84cbb947c8348b0b1f898 (table test-table [id=cb9b0caf1f414d26a720dd4f827a2962]) successfully deleted
I20260807 20:11:02.555239 19893 ts_tablet_manager.cc:1918] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:11:02.555299 19893 log.cc:1196] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/b93409b382d84cbb947c8348b0b1f898
I20260807 20:11:02.555552 19893 ts_tablet_manager.cc:1939] T b93409b382d84cbb947c8348b0b1f898 P 06cdb630a1174bdd951fdec8d77d1360: Deleting consensus metadata
I20260807 20:11:02.556030 19356 catalog_manager.cc:4841] TS 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): tablet b93409b382d84cbb947c8348b0b1f898 (table test-table [id=cb9b0caf1f414d26a720dd4f827a2962]) successfully deleted
W20260807 20:11:02.561615 19354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:37155 (error 108)
I20260807 20:11:02.561877 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19570
I20260807 20:11:02.562165 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:02.562020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:37155 (local address 127.0.0.1:38626)
0807 20:11:02.562073 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:02.562109 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 20:11:02.562296 19354 catalog_manager.cc:4584] TS 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b93409b382d84cbb947c8348b0b1f898: Network error: Client connection negotiation failed: client connection to 127.3.15.129:37155: connect: Connection refused (error 111)
I20260807 20:11:02.568648 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19714
I20260807 20:11:02.576082 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19336
I20260807 20:11:22.583448 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45179
--webserver_interface=127.3.15.190
--webserver_port=40461
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:45179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:11:22.669528 19895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:22.669744 19895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:22.669766 19895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:22.670928 19895 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 20:11:22.670976 19895 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 20:11:22.670990 19895 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 20:11:22.671038 19895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:11:22.671057 19895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:22.671082 19895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:22.671111 19895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:11:22.671161 19895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:11:22.672667 19895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:45179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=40461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:22.672943 19895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:22.673131 19895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:22.682586 19901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:22.682600 19902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:22.682590 19904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:22.682924 19895 server_base.cc:1034] running on GCE node
I20260807 20:11:22.683248 19895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:22.683561 19895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:22.684726 19895 hybrid_clock.cc:648] HybridClock initialized: now 1786133482684704 us; error 38 us; skew 500 ppm
Aug 07 20:11:22 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133482, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:22.687345 19895 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:11:22.687853 19895 webserver.cc:460] Webserver started at http://127.3.15.190:40461/ using document root <none> and password file <none>
I20260807 20:11:22.688102 19895 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:22.688194 19895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:22.689700 19895 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:22.690429 19911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:22.690615 19895 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:11:22.690729 19895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "25f3e79a9a054d6aad0b444522328245"
format_stamp: "Formatted at 2026-08-07 20:11:01 on dist-test-slave-1x4g"
server_key: "301da9b6b228c114a8dd3eba948e986b"
server_key_iv: "d1153c2ea044701b0a1acfac76b3f6ea"
server_key_version: "encryptionkey@0"
I20260807 20:11:22.691169 19895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:22.700299 19895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:22.702307 19895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:22.702474 19895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:22.706743 19895 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:45179
I20260807 20:11:22.706776 19965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:45179 every 8 connection(s)
I20260807 20:11:22.707229 19895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:11:22.708894 19966 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:11:22.709625 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19895
I20260807 20:11:22.709630 19966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap starting.
I20260807 20:11:22.709942 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:37155
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:45179
--webserver_port=41035
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:11:22.718622 19966 log.cc:824] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:22.720664 19966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:11:22.721032 19966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap complete.
I20260807 20:11:22.723371 19966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:22.723656 19966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f3e79a9a054d6aad0b444522328245, State: Initialized, Role: FOLLOWER
I20260807 20:11:22.723842 19966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:22.723907 19966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:11:22.723976 19966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:11:22.724027 19966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 1 FOLLOWER]: Advancing to term 2
I20260807 20:11:22.725302 19966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:22.725399 19966 leader_election.cc:302] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25f3e79a9a054d6aad0b444522328245; no voters:
I20260807 20:11:22.725630 19966 leader_election.cc:288] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 20:11:22.725672 19970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 20:11:22.725831 19970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 LEADER]: Becoming Leader. State: Replica: 25f3e79a9a054d6aad0b444522328245, State: Running, Role: LEADER
I20260807 20:11:22.725915 19966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:11:22.725932 19970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:22.726213 19970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25f3e79a9a054d6aad0b444522328245. Latest consensus state: current_term: 2 leader_uuid: "25f3e79a9a054d6aad0b444522328245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } } }
I20260807 20:11:22.726291 19970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: This master's current role is: LEADER
I20260807 20:11:22.726218 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "25f3e79a9a054d6aad0b444522328245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } } }
I20260807 20:11:22.726338 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: This master's current role is: LEADER
I20260807 20:11:22.726574 19975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:11:22.727474 19975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:11:22.727591 19975 catalog_manager.cc:1259] Loaded cluster ID: b2f7c1dae1a24300a2d897a33c8c0f75
I20260807 20:11:22.727628 19975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:11:22.728879 19975 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:11:22.729094 19975 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Loaded TSK: 0
I20260807 20:11:22.735126 19975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Generated new TSK 1
I20260807 20:11:22.735230 19975 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:11:22.806682 19968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:22.806918 19968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:22.806960 19968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:22.808125 19968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:22.808189 19968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:22.808238 19968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:11:22.809772 19968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:37155
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=41035
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:22.810038 19968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:22.810276 19968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:22.815502 19997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:22.815560 19996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:22.815560 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:22.816515 19968 server_base.cc:1034] running on GCE node
I20260807 20:11:22.816766 19968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:22.817085 19968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:22.818259 19968 hybrid_clock.cc:648] HybridClock initialized: now 1786133482818249 us; error 28 us; skew 500 ppm
Aug 07 20:11:22 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133482, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:22.820770 19968 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:11:22.821332 19968 webserver.cc:460] Webserver started at http://127.3.15.129:41035/ using document root <none> and password file <none>
I20260807 20:11:22.821594 19968 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:22.821653 19968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:22.823107 19968 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:22.823818 20007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:22.823999 19968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:22.824120 19968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "94ad79cfc59148399d7c0fabd4c25d39"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "46b3c78788b4db27d44d531ceb01de25"
server_key_iv: "35369cad9beb91b7efb91c41aa8d34be"
server_key_version: "encryptionkey@0"
I20260807 20:11:22.824491 19968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:22.854972 19968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:22.857337 19968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:22.857492 19968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:22.857795 19968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:22.858377 20014 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:11:22.859920 19968 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:11:22.859973 19968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:11:22.860009 19968 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:11:22.860715 19968 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:11:22.860761 19968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 20:11:22.860827 20014 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap starting.
I20260807 20:11:22.868834 19968 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:37155
I20260807 20:11:22.869071 20127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:37155 every 8 connection(s)
I20260807 20:11:22.869413 19968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:11:22.872469 20014 log.cc:824] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:22.875168 20014 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:11:22.875480 20014 tablet_bootstrap.cc:492] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap complete.
I20260807 20:11:22.876513 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 19968
I20260807 20:11:22.876886 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:42485
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:45179
--webserver_port=37399
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:11:22.876864 20014 ts_tablet_manager.cc:1397] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.013s
I20260807 20:11:22.878017 20014 raft_consensus.cc:348] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:22.878386 20014 raft_consensus.cc:729] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ad79cfc59148399d7c0fabd4c25d39, State: Initialized, Role: FOLLOWER
I20260807 20:11:22.878896 20014 consensus_queue.cc:260] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } }
I20260807 20:11:22.879206 20014 ts_tablet_manager.cc:1428] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
Aug 07 20:11:22 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133482, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:22.884651 20130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:22.870179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:39321)
0807 20:11:22.872509 (+ 2330us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:22.872584 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:22.873320 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:22.873833 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:22.873838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:22.874201 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:22.874484 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:22.874490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.875729 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.875731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:22.877216 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:22.877219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.878086 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.878087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:22.878227 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:22.878841 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:22.878852 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:22.880410 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:22.882838 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:22.882846 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:22.882848 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:22.883150 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:22.883498 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:22.883499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:22.883500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:22.883620 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:22.884136 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:22.884142 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:22.884264 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 20:11:22.884352 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 20:11:22.884490 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2285,"thread_start_us":91,"threads_started":1}
I20260807 20:11:22.884931 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:22.870305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:39321 (local address 127.3.15.190:45179)
0807 20:11:22.870536 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:22.870546 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:22.872728 (+ 2182us) server_negotiation.cc:407] Connection header received
0807 20:11:22.873563 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:22.873570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:22.873625 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:22.873748 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:22.874606 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.875608 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.877342 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.877981 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.878097 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:22.880523 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:22.880528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:22.880528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:22.880541 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:22.882727 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:22.883249 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:22.883254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:22.883254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:22.883393 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:22.883703 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:22.883704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:22.883704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:22.884058 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:22.884158 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:22.884448 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 20:11:22.884629 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
I20260807 20:11:22.885622 20128 heartbeater.cc:347] Connected to a master server at 127.3.15.190:45179
I20260807 20:11:22.885749 20128 heartbeater.cc:464] Registering TS with master...
I20260807 20:11:22.885931 20128 heartbeater.cc:511] Master 127.3.15.190:45179 requested a full tablet report, sending...
I20260807 20:11:22.886633 19930 ts_manager.cc:194] Registered new tserver with Master: 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155)
I20260807 20:11:22.887660 19930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:39321
I20260807 20:11:22.888517 20128 heartbeater.cc:503] Master 127.3.15.190:45179 was elected leader, sending a full tablet report...
I20260807 20:11:22.895838 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:22.887271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:37155 (local address 127.0.0.1:41484)
0807 20:11:22.892099 (+ 4828us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:22.892132 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:22.892201 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:22.892388 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:22.892390 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:22.892392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:22.892447 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:22.892448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.893728 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.893729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:22.894592 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:22.894595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.895598 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.895599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:22.895666 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:22.895678 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 20:11:22.895736 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:11:22.895778 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4796,"thread_start_us":91,"threads_started":1}
I20260807 20:11:22.895879 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:22.887340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.3.15.129:37155)
0807 20:11:22.887501 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:22.887506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:22.892217 (+ 4711us) server_negotiation.cc:407] Connection header received
0807 20:11:22.892268 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:22.892271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:22.892289 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:22.892380 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:22.892511 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.893656 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.894694 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:22.895510 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:22.895589 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:22.895654 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:22.895769 (+ 115us) server_negotiation.cc:299] Negotiation successful
0807 20:11:22.895806 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260807 20:11:22.896325 20058 tablet_service.cc:1514] Processing DeleteTablet for tablet b93409b382d84cbb947c8348b0b1f898 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:02 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41484
I20260807 20:11:22.897022 20137 tablet_replica.cc:330] stopping tablet replica
I20260807 20:11:22.897166 20137 raft_consensus.cc:2229] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:11:22.897255 20137 raft_consensus.cc:2258] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:11:22.897588 20137 ts_tablet_manager.cc:1905] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:11:22.899895 20137 ts_tablet_manager.cc:1918] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:11:22.900000 20137 log.cc:1196] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal/wals/b93409b382d84cbb947c8348b0b1f898
I20260807 20:11:22.900249 20137 ts_tablet_manager.cc:1939] T b93409b382d84cbb947c8348b0b1f898 P 94ad79cfc59148399d7c0fabd4c25d39: Deleting consensus metadata
I20260807 20:11:22.900810 19913 catalog_manager.cc:4841] TS 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155): tablet b93409b382d84cbb947c8348b0b1f898 (table test-table [id=cb9b0caf1f414d26a720dd4f827a2962]) successfully deleted
W20260807 20:11:22.965134 20133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:22.965389 20133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:22.965428 20133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:22.966648 20133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:22.966691 20133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:22.966717 20133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:11:22.968176 20133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:42485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=37399
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:22.968410 20133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:22.968665 20133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:22.973801 20145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:22.973839 20143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:22.973991 20142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:22.974007 20133 server_base.cc:1034] running on GCE node
I20260807 20:11:22.974257 20133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:22.974512 20133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:22.975692 20133 hybrid_clock.cc:648] HybridClock initialized: now 1786133482975684 us; error 39 us; skew 500 ppm
Aug 07 20:11:22 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133482, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:22.978334 20133 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:11:22.978792 20133 webserver.cc:460] Webserver started at http://127.3.15.130:37399/ using document root <none> and password file <none>
I20260807 20:11:22.979039 20133 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:22.979126 20133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:22.980531 20133 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:11:22.981299 20153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:22.981482 20133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:11:22.981587 20133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "1f47d3a4984a4ae78b77fecc6abdbb18"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "1ca1a921bbf4cba5a431e6e2cb9dcaee"
server_key_iv: "5e00aa14d140d7cdb89fc3b5619d46f0"
server_key_version: "encryptionkey@0"
I20260807 20:11:22.981984 20133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:23.000066 20133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:23.002163 20133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:23.002337 20133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:23.002586 20133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:23.002914 20133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:23.002974 20133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:23.003008 20133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:23.003054 20133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:23.008874 20133 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:42485
I20260807 20:11:23.008955 20272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:42485 every 8 connection(s)
I20260807 20:11:23.009353 20133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:11:23.013835 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 20133
I20260807 20:11:23.014161 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:34547
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:45179
--webserver_port=38875
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 20:11:23 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133482, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:23.019801 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.010195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:39715 (local address 127.3.15.190:45179)
0807 20:11:23.010285 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.010293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.010510 (+ 217us) server_negotiation.cc:407] Connection header received
0807 20:11:23.011006 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.011008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.011022 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.011102 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:23.012207 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.012702 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.013769 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.014115 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.014202 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.016725 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:23.016731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:23.016732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:23.016739 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:23.018296 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.018751 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.018753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.018753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.018819 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.019102 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.019103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.019104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.019229 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:23.019302 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.019601 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.019707 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 20:11:23.020000 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.010170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:39715)
0807 20:11:23.010344 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.010390 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:23.010897 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:23.011163 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:23.011166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:23.011890 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:23.012117 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.012123 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.012832 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.012833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.013678 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.013680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.014209 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.014226 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.014396 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.014971 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:23.014980 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:23.016591 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:23.018403 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:23.018411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:23.018414 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:23.018668 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:23.018891 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:23.018891 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:23.018892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:23.019014 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:23.019308 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:23.019316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:23.019462 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 20:11:23.019575 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 20:11:23.019745 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
I20260807 20:11:23.020531 20273 heartbeater.cc:347] Connected to a master server at 127.3.15.190:45179
I20260807 20:11:23.020640 20273 heartbeater.cc:464] Registering TS with master...
I20260807 20:11:23.020892 20273 heartbeater.cc:511] Master 127.3.15.190:45179 requested a full tablet report, sending...
I20260807 20:11:23.021344 19930 ts_manager.cc:194] Registered new tserver with Master: 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485)
I20260807 20:11:23.021744 19930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:39715
W20260807 20:11:23.100080 20276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:23.100332 20276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:23.100373 20276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:23.101572 20276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:23.101666 20276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:23.101714 20276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:11:23.103242 20276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:34547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=38875
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:23.103521 20276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:23.103771 20276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:23.108973 20285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:23.109000 20281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:23.108996 20282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:23.109057 20276 server_base.cc:1034] running on GCE node
I20260807 20:11:23.109390 20276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:23.109639 20276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:23.110809 20276 hybrid_clock.cc:648] HybridClock initialized: now 1786133483110780 us; error 51 us; skew 500 ppm
Aug 07 20:11:23 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133483, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:23.113430 20276 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:11:23.113888 20276 webserver.cc:460] Webserver started at http://127.3.15.131:38875/ using document root <none> and password file <none>
I20260807 20:11:23.114138 20276 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:23.114220 20276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:23.115586 20276 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:23.116178 20294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:23.116353 20276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:23.116469 20276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "06cdb630a1174bdd951fdec8d77d1360"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "9626cf1d065cfaf8d74388758242615b"
server_key_iv: "b323563e01c71785a876dfff62382d9d"
server_key_version: "encryptionkey@0"
I20260807 20:11:23.116918 20276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:23.130296 20276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:23.132405 20276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:23.132577 20276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:23.132884 20276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:23.133227 20276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:23.133282 20276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:23.133319 20276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:23.133347 20276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:23.139410 20276 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:34547
I20260807 20:11:23.139463 20413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:34547 every 8 connection(s)
I20260807 20:11:23.139822 20276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:11:23.140321 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 20276
Aug 07 20:11:23 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133483, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:11:23.148250 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.140862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41898 (local address 127.3.15.190:45179)
0807 20:11:23.140933 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.140936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.141090 (+ 154us) server_negotiation.cc:407] Connection header received
0807 20:11:23.141146 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.141147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.141160 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.141219 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:23.141989 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.142820 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.143794 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.143998 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.144082 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.145444 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:23.145445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:23.145446 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:23.145452 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:23.147018 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.147415 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.147416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.147416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.147467 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.147690 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.147691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.147691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.147790 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:23.147907 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.148008 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.148134 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:11:23.150619 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.141078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:51317 (local address 127.3.15.190:45179)
0807 20:11:23.141191 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.141193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.141465 (+ 272us) server_negotiation.cc:407] Connection header received
0807 20:11:23.141946 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.141948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.141963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.142088 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:23.143304 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.143951 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.145233 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.145518 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.145568 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.148099 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:23.148100 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:23.148100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:23.148107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:23.149484 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.149847 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.149847 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.149848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.149898 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.150073 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.150073 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.150073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.150165 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:23.150214 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.150468 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.150532 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
I20260807 20:11:23.150790 20418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.141146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:51317)
0807 20:11:23.141334 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.141382 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:23.141847 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:23.142236 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:23.142241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:23.143007 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:23.143211 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.143217 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.144035 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.144036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.145148 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.145151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.145616 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.145617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.145772 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.146168 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:23.146179 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:23.148010 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:23.149556 (+ 1546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:23.149561 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:23.149563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:23.149781 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:23.149939 (+ 158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:23.149940 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:23.149940 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:23.150023 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:23.150235 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:23.150240 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:23.150357 (+ 117us) client_negotiation.cc:769] Sending connection context
0807 20:11:23.150442 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 20:11:23.150563 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260807 20:11:23.151338 20414 heartbeater.cc:347] Connected to a master server at 127.3.15.190:45179
I20260807 20:11:23.151433 20414 heartbeater.cc:464] Registering TS with master...
I20260807 20:11:23.151621 20414 heartbeater.cc:511] Master 127.3.15.190:45179 requested a full tablet report, sending...
I20260807 20:11:23.152043 19930 ts_manager.cc:194] Registered new tserver with Master: 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
I20260807 20:11:23.152412 19930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:51317
I20260807 20:11:23.160187 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:11:23.162873 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.160570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41902 (local address 127.3.15.190:45179)
0807 20:11:23.160640 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.160642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.160733 (+ 91us) server_negotiation.cc:407] Connection header received
0807 20:11:23.160817 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.160817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.160865 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.160931 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:11:23.161054 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.161502 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.162233 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.162413 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.162463 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.162653 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:11:23.162695 (+ 42us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 20:11:23.162826 (+ 131us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41902: token expired
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 20:11:23.162943 20417 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41902: token expired
W20260807 20:11:23.163830 19856 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.15.190:45179)
I20260807 20:11:23.169513 20417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.164105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.3.15.190:45179)
0807 20:11:23.164137 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.164139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.164170 (+ 31us) server_negotiation.cc:407] Connection header received
0807 20:11:23.164195 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.164196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.164204 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.164239 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:23.164676 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.165172 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.165816 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.166023 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.166076 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.167164 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:23.167165 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:23.167165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:23.167171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:23.168561 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.168934 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.168935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.168935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.168985 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:23.169113 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:23.169113 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:23.169113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:23.169226 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:23.169294 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.169382 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.169439 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260807 20:11:23.170876 19930 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:11:23.172564 19930 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:11:23.174985 20058 tablet_service.cc:1467] Processing CreateTablet for tablet fb707b0b202c4497909182f2b69befd4 (DEFAULT_TABLE table=test-table [id=469eb171f32e45ca9307925098c349d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:11:23.175210 20058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb707b0b202c4497909182f2b69befd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:23.176817 20014 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap starting.
I20260807 20:11:23.177178 20014 tablet_bootstrap.cc:654] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:23.177768 20014 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: No bootstrap required, opened a new log
I20260807 20:11:23.177809 20014 ts_tablet_manager.cc:1397] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:23.177918 20014 raft_consensus.cc:348] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.178004 20014 raft_consensus.cc:374] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:23.178058 20014 raft_consensus.cc:729] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ad79cfc59148399d7c0fabd4c25d39, State: Initialized, Role: FOLLOWER
I20260807 20:11:23.178113 20014 consensus_queue.cc:260] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.178201 20014 ts_tablet_manager.cc:1428] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 20:11:23.179186 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.174727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:34547 (local address 127.0.0.1:40310)
0807 20:11:23.174894 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.174903 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:23.174944 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:23.175379 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:23.175381 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:23.175383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:23.175436 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.175438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.176732 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.176734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.177787 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.177791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.178936 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.178937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.179009 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.179013 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:11:23.179063 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:11:23.179109 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260807 20:11:23.179239 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.175003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.3.15.131:34547)
0807 20:11:23.175230 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.175234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.175243 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:11:23.175279 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.175281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.175296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.175373 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:23.175512 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.176607 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.177868 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.178863 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.178936 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.179002 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.179080 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.179136 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":132,"threads_started":1}
I20260807 20:11:23.179250 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.174886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.3.15.130:42485)
0807 20:11:23.175108 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.175112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.175121 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:11:23.175182 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.175184 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.175201 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.175449 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:23.175542 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.176753 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.178038 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.178903 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.178983 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.179041 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.179133 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.179171 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260807 20:11:23.179293 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.174692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42485 (local address 127.0.0.1:51410)
0807 20:11:23.174741 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.174752 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:23.174787 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:23.175321 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:23.175322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:23.175324 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:23.175371 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.175372 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.176871 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.176872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.177869 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.177872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.178993 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.178994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.179057 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.179061 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:11:23.179118 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 20:11:23.179164 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 20:11:23.180299 20204 tablet_service.cc:1467] Processing CreateTablet for tablet fb707b0b202c4497909182f2b69befd4 (DEFAULT_TABLE table=test-table [id=469eb171f32e45ca9307925098c349d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:11:23.180665 20204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb707b0b202c4497909182f2b69befd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:23.183657 20424 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Bootstrap starting.
I20260807 20:11:23.184083 20345 tablet_service.cc:1467] Processing CreateTablet for tablet fb707b0b202c4497909182f2b69befd4 (DEFAULT_TABLE table=test-table [id=469eb171f32e45ca9307925098c349d0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:11:23.184370 20424 tablet_bootstrap.cc:654] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:23.184454 20345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb707b0b202c4497909182f2b69befd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:11:23.184974 20424 log.cc:824] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:23.185654 20424 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: No bootstrap required, opened a new log
I20260807 20:11:23.185708 20424 ts_tablet_manager.cc:1397] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:11:23.187067 20424 raft_consensus.cc:348] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.187206 20424 raft_consensus.cc:374] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:23.187261 20424 raft_consensus.cc:729] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f47d3a4984a4ae78b77fecc6abdbb18, State: Initialized, Role: FOLLOWER
I20260807 20:11:23.187384 20424 consensus_queue.cc:260] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.187650 20427 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Bootstrap starting.
I20260807 20:11:23.187706 20424 ts_tablet_manager.cc:1428] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:11:23.187826 20273 heartbeater.cc:503] Master 127.3.15.190:45179 was elected leader, sending a full tablet report...
I20260807 20:11:23.188357 20427 tablet_bootstrap.cc:654] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Neither blocks nor log segments found. Creating new log.
I20260807 20:11:23.188912 20427 log.cc:824] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:23.189610 20427 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: No bootstrap required, opened a new log
I20260807 20:11:23.189666 20427 ts_tablet_manager.cc:1397] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:11:23.190884 20427 raft_consensus.cc:348] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.191012 20427 raft_consensus.cc:374] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:11:23.191035 20427 raft_consensus.cc:729] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06cdb630a1174bdd951fdec8d77d1360, State: Initialized, Role: FOLLOWER
I20260807 20:11:23.191148 20427 consensus_queue.cc:260] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.191380 20427 ts_tablet_manager.cc:1428] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 20:11:23.191439 20414 heartbeater.cc:503] Master 127.3.15.190:45179 was elected leader, sending a full tablet report...
I20260807 20:11:23.218742 20428 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:23.218863 20428 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.219355 20428 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547), 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155)
I20260807 20:11:23.223296 20136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.219746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:38359 (local address 127.3.15.129:37155)
0807 20:11:23.219819 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.219821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.219850 (+ 29us) server_negotiation.cc:407] Connection header received
0807 20:11:23.219875 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.219876 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.219887 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.219952 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:23.220066 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.220808 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.222331 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.222899 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.222981 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.222990 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.223183 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.223224 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 20:11:23.223296 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.219618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46159 (local address 127.3.15.131:34547)
0807 20:11:23.219692 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.219694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.219703 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:11:23.219743 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.219744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.219753 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.219808 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:11:23.219970 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.220759 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.222332 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.222906 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.222981 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.222990 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.223181 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.223224 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:11:23.223325 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.219531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:34547 (local address 127.3.15.130:46159)
0807 20:11:23.219589 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.219600 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:23.219675 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:23.219818 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:23.219820 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:23.219821 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:23.219898 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.219899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.220869 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.220870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.222233 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.222237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.222991 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.222992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.223094 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.223098 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:11:23.223161 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:11:23.223224 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":89}
I20260807 20:11:23.223330 20431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.219544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:37155 (local address 127.3.15.130:38359)
0807 20:11:23.219753 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.219764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:23.219818 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:23.219940 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:23.219941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:23.219943 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:11:23.219998 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.219999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.220878 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.220878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.222233 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:23.222237 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.222992 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.222992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:23.223096 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.223099 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:11:23.223161 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 20:11:23.223224 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":47,"thread_start_us":135,"threads_started":1}
I20260807 20:11:23.223739 20366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb707b0b202c4497909182f2b69befd4" candidate_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06cdb630a1174bdd951fdec8d77d1360" is_pre_election: true
I20260807 20:11:23.223848 20080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb707b0b202c4497909182f2b69befd4" candidate_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ad79cfc59148399d7c0fabd4c25d39" is_pre_election: true
I20260807 20:11:23.223929 20366 raft_consensus.cc:2454] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f47d3a4984a4ae78b77fecc6abdbb18 in term 0.
I20260807 20:11:23.224028 20080 raft_consensus.cc:2454] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f47d3a4984a4ae78b77fecc6abdbb18 in term 0.
I20260807 20:11:23.224159 20157 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18; no voters:
I20260807 20:11:23.224308 20428 raft_consensus.cc:2781] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:11:23.224377 20428 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:11:23.224396 20428 raft_consensus.cc:3037] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:23.225111 20428 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.225258 20428 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [CANDIDATE]: Term 1 election: Requested vote from peers 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547), 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155)
I20260807 20:11:23.225416 20366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb707b0b202c4497909182f2b69befd4" candidate_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06cdb630a1174bdd951fdec8d77d1360"
I20260807 20:11:23.225462 20080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb707b0b202c4497909182f2b69befd4" candidate_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94ad79cfc59148399d7c0fabd4c25d39"
I20260807 20:11:23.225539 20366 raft_consensus.cc:3037] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:23.225526 20080 raft_consensus.cc:3037] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:11:23.226405 20366 raft_consensus.cc:2454] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f47d3a4984a4ae78b77fecc6abdbb18 in term 1.
I20260807 20:11:23.226716 20157 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18; no voters:
I20260807 20:11:23.226809 20428 raft_consensus.cc:2781] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:11:23.226986 20428 raft_consensus.cc:686] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 LEADER]: Becoming Leader. State: Replica: 1f47d3a4984a4ae78b77fecc6abdbb18, State: Running, Role: LEADER
I20260807 20:11:23.226990 20080 raft_consensus.cc:2454] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f47d3a4984a4ae78b77fecc6abdbb18 in term 1.
I20260807 20:11:23.227095 20428 consensus_queue.cc:237] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:23.228065 19930 catalog_manager.cc:5495] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } health_report { overall_health: UNKNOWN } } }
I20260807 20:11:23.239759 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.237330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.3.15.130:42485)
0807 20:11:23.237392 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:23.237396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:23.237409 (+ 13us) server_negotiation.cc:407] Connection header received
0807 20:11:23.237441 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:23.237441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:23.237460 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:23.237537 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:11:23.237672 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.238164 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.238927 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:23.239124 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:23.239201 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:23.239386 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:11:23.239495 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:11:23.239589 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:23.239628 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:11:23.239650 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:11:23.241861 20366 raft_consensus.cc:1261] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Refusing update from remote peer 1f47d3a4984a4ae78b77fecc6abdbb18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:11:23.241945 20080 raft_consensus.cc:1261] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Refusing update from remote peer 1f47d3a4984a4ae78b77fecc6abdbb18: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:11:23.242352 20428 consensus_queue.cc:1035] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:11:23.242480 20432 consensus_queue.cc:1035] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:11:23.245128 20436 mvcc.cc:205] Tried to move back new op lower bound from 7316002747355635712 to 7316002747299192832. Current Snapshot: MvccSnapshot[applied={T|T < 7316002747355635712}]
I20260807 20:11:23.245325 20437 mvcc.cc:205] Tried to move back new op lower bound from 7316002747355635712 to 7316002747299192832. Current Snapshot: MvccSnapshot[applied={T|T < 7316002747355635712}]
I20260807 20:11:23.245473 20440 mvcc.cc:205] Tried to move back new op lower bound from 7316002747355635712 to 7316002747299192832. Current Snapshot: MvccSnapshot[applied={T|T < 7316002747355635712}]
I20260807 20:11:23.247516 19928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:11:23.247635 19928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:11:23.248449 19928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Sending DeleteTablet for 3 replicas of tablet fb707b0b202c4497909182f2b69befd4
I20260807 20:11:23.248821 20058 tablet_service.cc:1514] Processing DeleteTablet for tablet fb707b0b202c4497909182f2b69befd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:23 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41484
I20260807 20:11:23.248839 20204 tablet_service.cc:1514] Processing DeleteTablet for tablet fb707b0b202c4497909182f2b69befd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:23 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:51410
I20260807 20:11:23.248903 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19968
I20260807 20:11:23.249153 20345 tablet_service.cc:1514] Processing DeleteTablet for tablet fb707b0b202c4497909182f2b69befd4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:11:23 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:40310
I20260807 20:11:23.251600 20448 tablet_replica.cc:330] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: stopping tablet replica
I20260807 20:11:23.251752 20448 raft_consensus.cc:2229] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:11:23.251920 20448 raft_consensus.cc:2258] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:11:23.253304 20448 ts_tablet_manager.cc:1905] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:11:23.255170 20448 ts_tablet_manager.cc:1918] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:11:23.255261 20448 log.cc:1196] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/fb707b0b202c4497909182f2b69befd4
I20260807 20:11:23.255568 20448 ts_tablet_manager.cc:1939] T fb707b0b202c4497909182f2b69befd4 P 1f47d3a4984a4ae78b77fecc6abdbb18: Deleting consensus metadata
I20260807 20:11:23.256072 19912 catalog_manager.cc:4841] TS 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): tablet fb707b0b202c4497909182f2b69befd4 (table test-table [id=469eb171f32e45ca9307925098c349d0]) successfully deleted
W20260807 20:11:23.257393 19913 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:37155 (error 108)
I20260807 20:11:23.257732 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 20133
I20260807 20:11:23.258912 20449 tablet_replica.cc:330] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: stopping tablet replica
I20260807 20:11:23.259076 20449 raft_consensus.cc:2229] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:11:23.259229 20449 raft_consensus.cc:2258] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:11:23.259959 20135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:23.257834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:37155 (local address 127.0.0.1:41494)
0807 20:11:23.259863 (+ 2029us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:23.259919 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:37155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2001}
I20260807 20:11:23.260272 20449 ts_tablet_manager.cc:1905] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 20:11:23.261013 19913 catalog_manager.cc:4584] TS 94ad79cfc59148399d7c0fabd4c25d39 (127.3.15.129:37155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb707b0b202c4497909182f2b69befd4: Network error: Client connection negotiation failed: client connection to 127.3.15.129:37155: connect: Connection refused (error 111)
I20260807 20:11:23.262701 20449 ts_tablet_manager.cc:1918] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:11:23.262784 20449 log.cc:1196] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/fb707b0b202c4497909182f2b69befd4
I20260807 20:11:23.263142 20449 ts_tablet_manager.cc:1939] T fb707b0b202c4497909182f2b69befd4 P 06cdb630a1174bdd951fdec8d77d1360: Deleting consensus metadata
I20260807 20:11:23.263615 19915 catalog_manager.cc:4841] TS 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): tablet fb707b0b202c4497909182f2b69befd4 (table test-table [id=469eb171f32e45ca9307925098c349d0]) successfully deleted
I20260807 20:11:23.266258 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 20276
I20260807 20:11:23.272413 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 19895
I20260807 20:11:43.279492 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45179
--webserver_interface=127.3.15.190
--webserver_port=40461
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:45179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:11:43.363826 20451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:43.364118 20451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:43.364158 20451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:43.365397 20451 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 20:11:43.365471 20451 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 20:11:43.365491 20451 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 20:11:43.365511 20451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:11:43.365538 20451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:43.365567 20451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:43.365595 20451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:11:43.365624 20451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:11:43.367313 20451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:45179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=40461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:43.367630 20451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:43.367884 20451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:43.373169 20456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:43.373236 20451 server_base.cc:1034] running on GCE node
W20260807 20:11:43.373230 20457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:43.373251 20459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:43.373674 20451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:43.373982 20451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:43.375136 20451 hybrid_clock.cc:648] HybridClock initialized: now 1786133503375123 us; error 39 us; skew 500 ppm
Aug 07 20:11:43 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133503, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:43.377717 20451 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:11:43.378197 20451 webserver.cc:460] Webserver started at http://127.3.15.190:40461/ using document root <none> and password file <none>
I20260807 20:11:43.378441 20451 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:43.378490 20451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:43.379904 20451 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.380645 20467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.380873 20451 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.380976 20451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "25f3e79a9a054d6aad0b444522328245"
format_stamp: "Formatted at 2026-08-07 20:11:01 on dist-test-slave-1x4g"
server_key: "301da9b6b228c114a8dd3eba948e986b"
server_key_iv: "d1153c2ea044701b0a1acfac76b3f6ea"
server_key_version: "encryptionkey@0"
I20260807 20:11:43.381350 20451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:43.389757 20451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:43.391549 20451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:43.391733 20451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:43.396265 20451 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:45179
I20260807 20:11:43.396296 20521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:45179 every 8 connection(s)
I20260807 20:11:43.396718 20451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:11:43.398264 20522 sys_catalog.cc:263] Verifying existing consensus state
I20260807 20:11:43.398896 20522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap starting.
I20260807 20:11:43.405479 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 20451
I20260807 20:11:43.405784 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:37155
--local_ip_for_outbound_sockets=127.3.15.129
--tserver_master_addrs=127.3.15.190:45179
--webserver_port=41035
--webserver_interface=127.3.15.129
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:11:43.407629 20522 log.cc:824] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:43.410008 20522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:11:43.410331 20522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Bootstrap complete.
I20260807 20:11:43.412612 20522 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:43.412926 20522 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25f3e79a9a054d6aad0b444522328245, State: Initialized, Role: FOLLOWER
I20260807 20:11:43.413110 20522 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:43.413169 20522 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:11:43.413225 20522 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:11:43.413275 20522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 2 FOLLOWER]: Advancing to term 3
I20260807 20:11:43.414299 20522 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:43.414389 20522 leader_election.cc:302] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25f3e79a9a054d6aad0b444522328245; no voters:
I20260807 20:11:43.414608 20522 leader_election.cc:288] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 20:11:43.414635 20526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 20:11:43.414762 20526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [term 3 LEADER]: Becoming Leader. State: Replica: 25f3e79a9a054d6aad0b444522328245, State: Running, Role: LEADER
I20260807 20:11:43.414853 20526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } }
I20260807 20:11:43.414907 20522 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:11:43.415159 20527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "25f3e79a9a054d6aad0b444522328245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } } }
I20260807 20:11:43.415407 20527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: This master's current role is: LEADER
I20260807 20:11:43.415536 20528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25f3e79a9a054d6aad0b444522328245. Latest consensus state: current_term: 3 leader_uuid: "25f3e79a9a054d6aad0b444522328245" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25f3e79a9a054d6aad0b444522328245" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45179 } } }
I20260807 20:11:43.415832 20536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:11:43.415840 20528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245 [sys.catalog]: This master's current role is: LEADER
I20260807 20:11:43.416834 20536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:11:43.416962 20536 catalog_manager.cc:1259] Loaded cluster ID: b2f7c1dae1a24300a2d897a33c8c0f75
I20260807 20:11:43.417222 20536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:11:43.418047 20536 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:11:43.418257 20536 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Loaded TSK: 0
I20260807 20:11:43.418315 20536 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 25f3e79a9a054d6aad0b444522328245: Loaded TSK: 1
F20260807 20:11:43.418552 20536 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f32528179ad google::LogMessage::Fail() at ??:0
@ 0x7f325281a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f32528174de google::LogMessage::Flush() at ??:0
@ 0x7f3252818a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3254f84104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3254f81d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f325a7751cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f325a77548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f325a775594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f325a73bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f325a745ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f32546e50d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f32546da954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f325553e6db start_thread at ??:0
@ 0x7f325131561f clone at ??:0
W20260807 20:11:43.507938 20524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:43.508159 20524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:43.508194 20524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:43.509407 20524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:43.509477 20524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:43.509523 20524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:11:43.511004 20524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:37155
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=41035
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:43.511271 20524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:43.511533 20524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:43.517838 20567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:43.518018 20524 server_base.cc:1034] running on GCE node
W20260807 20:11:43.517854 20568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:43.518177 20571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:43.518389 20524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:43.518647 20524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:43.519814 20524 hybrid_clock.cc:648] HybridClock initialized: now 1786133503519801 us; error 32 us; skew 500 ppm
Aug 07 20:11:43 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133503, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:43.522553 20524 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:11:43.523082 20524 webserver.cc:460] Webserver started at http://127.3.15.129:41035/ using document root <none> and password file <none>
I20260807 20:11:43.523332 20524 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:43.523401 20524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:43.524873 20524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.525597 20578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.525779 20524 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.525870 20524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "94ad79cfc59148399d7c0fabd4c25d39"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "46b3c78788b4db27d44d531ceb01de25"
server_key_iv: "35369cad9beb91b7efb91c41aa8d34be"
server_key_version: "encryptionkey@0"
I20260807 20:11:43.526232 20524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:43.534399 20524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:43.536680 20524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:43.536890 20524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:43.537144 20524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:43.537639 20586 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 20:11:43.539126 20524 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 20:11:43.539182 20524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 20:11:43.539208 20524 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 20:11:43.539884 20524 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 20:11:43.539929 20524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 20:11:43.539991 20586 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap starting.
I20260807 20:11:43.547904 20524 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:37155
I20260807 20:11:43.548493 20524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:11:43.551303 20586 log.cc:824] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Log is configured to *not* fsync() on all Append() calls
I20260807 20:11:43.551602 20698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:37155 every 8 connection(s)
I20260807 20:11:43.552076 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 20524
I20260807 20:11:43.552386 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:42485
--local_ip_for_outbound_sockets=127.3.15.130
--tserver_master_addrs=127.3.15.190:45179
--webserver_port=37399
--webserver_interface=127.3.15.130
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:11:43.554574 20586 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 20:11:43.554872 20586 tablet_bootstrap.cc:492] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Bootstrap complete.
I20260807 20:11:43.555909 20586 ts_tablet_manager.cc:1397] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.004s
I20260807 20:11:43.557214 20586 raft_consensus.cc:348] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:43.557603 20586 raft_consensus.cc:729] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94ad79cfc59148399d7c0fabd4c25d39, State: Initialized, Role: FOLLOWER
I20260807 20:11:43.557780 20586 consensus_queue.cc:260] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:43.558082 20586 ts_tablet_manager.cc:1428] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
W20260807 20:11:43.624577 20701 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:11:43.549311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:44967)
0807 20:11:43.549513 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.549558 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:43.550037 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:43.624340 (+ 74303us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
W20260807 20:11:43.624892 20699 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:45179 (0 consecutive failures): Network error: Failed to ping master at 127.3.15.190:45179: Client connection negotiation failed: client connection to 127.3.15.190:45179: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 20:11:43.625226 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.625105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:39303)
0807 20:11:43.625162 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.625201 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:11:43.625624 20701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.625536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:50467)
0807 20:11:43.625586 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.625609 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 20:11:43.625756 20699 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:11:43.640039 20706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:43.640266 20706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:43.640308 20706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:43.641561 20706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:43.641623 20706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:43.641671 20706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:11:43.643232 20706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:42485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=37399
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:43.643503 20706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:43.643743 20706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:11:43.648880 20712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:43.648936 20713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:43.648895 20715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:43.649091 20706 server_base.cc:1034] running on GCE node
I20260807 20:11:43.649391 20706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:43.649649 20706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:43.650810 20706 hybrid_clock.cc:648] HybridClock initialized: now 1786133503650799 us; error 28 us; skew 500 ppm
Aug 07 20:11:43 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133503, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:43.653451 20706 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:11:43.653923 20706 webserver.cc:460] Webserver started at http://127.3.15.130:37399/ using document root <none> and password file <none>
I20260807 20:11:43.654155 20706 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:43.654201 20706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:43.655530 20706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.656114 20723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.656284 20706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:11:43.656370 20706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "1f47d3a4984a4ae78b77fecc6abdbb18"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "1ca1a921bbf4cba5a431e6e2cb9dcaee"
server_key_iv: "5e00aa14d140d7cdb89fc3b5619d46f0"
server_key_version: "encryptionkey@0"
I20260807 20:11:43.656713 20706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:43.669835 20706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:43.671690 20706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:43.671804 20706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:43.671995 20706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:43.672262 20706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:43.672293 20706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.672320 20706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:43.672333 20706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.678313 20706 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:42485
I20260807 20:11:43.678398 20843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:42485 every 8 connection(s)
I20260807 20:11:43.678766 20706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:11:43.679822 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.679434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:44625)
0807 20:11:43.679614 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.679684 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
W20260807 20:11:43.680255 20844 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:45179 (0 consecutive failures): Network error: Failed to ping master at 127.3.15.190:45179: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
I20260807 20:11:43.680569 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.680464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:52915)
0807 20:11:43.680517 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.680549 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:11:43.680634 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 20706
I20260807 20:11:43.680932 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:34547
--local_ip_for_outbound_sockets=127.3.15.131
--tserver_master_addrs=127.3.15.190:45179
--webserver_port=38875
--webserver_interface=127.3.15.131
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:11:43.680953 20847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.680862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:39089)
0807 20:11:43.680906 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.680930 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:11:43.681056 20844 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 20:11:43.764971 20848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:11:43.765177 20848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:11:43.765199 20848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:11:43.766433 20848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:11:43.766476 20848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:11:43.766502 20848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:11:43.767985 20848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:34547
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=38875
--tserver_master_addrs=127.3.15.190:45179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:11:43.768245 20848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:11:43.768478 20848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:11:43.773628 20848 server_base.cc:1034] running on GCE node
W20260807 20:11:43.773591 20854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:43.773641 20853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:11:43.773573 20857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:11:43.773947 20848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:11:43.774196 20848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:11:43.775357 20848 hybrid_clock.cc:648] HybridClock initialized: now 1786133503775345 us; error 28 us; skew 500 ppm
Aug 07 20:11:43 dist-test-slave-1x4g krb5kdc[19322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133503, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:11:43.778124 20848 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:11:43.778578 20848 webserver.cc:460] Webserver started at http://127.3.15.131:38875/ using document root <none> and password file <none>
I20260807 20:11:43.778767 20848 fs_manager.cc:359] Metadata directory not provided
I20260807 20:11:43.778808 20848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:11:43.780056 20848 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.780635 20864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.780850 20848 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:11:43.780953 20848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "06cdb630a1174bdd951fdec8d77d1360"
format_stamp: "Formatted at 2026-08-07 20:11:02 on dist-test-slave-1x4g"
server_key: "9626cf1d065cfaf8d74388758242615b"
server_key_iv: "b323563e01c71785a876dfff62382d9d"
server_key_version: "encryptionkey@0"
I20260807 20:11:43.781342 20848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:11:43.794317 20848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:11:43.796710 20848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:11:43.796903 20848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:11:43.797153 20848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:11:43.797466 20848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:11:43.797523 20848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.797567 20848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:11:43.797591 20848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:11:43.803756 20848 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:34547
I20260807 20:11:43.803866 20983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:34547 every 8 connection(s)
I20260807 20:11:43.804199 20848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:11:43.805346 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.804928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:39659)
0807 20:11:43.805093 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.805194 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
W20260807 20:11:43.805770 20984 heartbeater.cc:644] Failed to heartbeat to 127.3.15.190:45179 (0 consecutive failures): Network error: Failed to ping master at 127.3.15.190:45179: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
I20260807 20:11:43.806030 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.805942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:52435)
0807 20:11:43.805986 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.806012 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 20:11:43.806290 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:43.806239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:39419)
0807 20:11:43.806261 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:43.806279 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260807 20:11:43.806399 20984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 20:11:43.806687 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 20848
I20260807 20:11:44.626420 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:44.626164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:57685)
0807 20:11:44.626310 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:44.626352 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260807 20:11:44.681555 20990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:44.681339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:38465)
0807 20:11:44.681469 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:44.681501 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260807 20:11:44.806924 20991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:44.806716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:35345)
0807 20:11:44.806840 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:44.806872 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260807 20:11:45.019593 20992 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:45.019721 20992 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:45.020118 20992 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
Aug 07 20:11:45 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133503, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.130@KRBTEST.COM
Aug 07 20:11:45 dist-test-slave-1x4g krb5kdc[19322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133503, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.131@KRBTEST.COM
I20260807 20:11:45.031087 20996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.020511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:43177 (local address 127.3.15.131:34547)
0807 20:11:45.020651 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:45.020660 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:45.020688 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:11:45.021171 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:45.021174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:45.021219 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:45.021372 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:45.022153 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.022993 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.023983 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.024376 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.024491 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:45.027360 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:45.027365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:45.027365 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:45.027405 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:45.029169 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:45.029828 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:45.029831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:45.029831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:45.029936 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:45.030206 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:45.030207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:45.030208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:45.030447 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:45.030540 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:45.030826 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 20:11:45.030962 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260807 20:11:45.031073 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.020307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:34547 (local address 127.3.15.129:43177)
0807 20:11:45.020377 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:45.020392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:45.020468 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:45.021403 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:45.021406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:45.021810 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:45.022012 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:45.022017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.023100 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.023101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:45.023883 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:45.023885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.024477 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.024478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:45.024606 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:45.025210 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:45.025219 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:45.027238 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:45.029265 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:45.029273 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:45.029275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:45.029742 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:45.030020 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:45.030021 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:45.030021 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:45.030135 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:45.030523 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:45.030530 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:45.030658 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 20:11:45.030769 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 20:11:45.030966 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":14}
I20260807 20:11:45.031104 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.020433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:42747 (local address 127.3.15.130:42485)
0807 20:11:45.020604 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 20:11:45.020618 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 20:11:45.020656 (+ 38us) server_negotiation.cc:407] Connection header received
0807 20:11:45.021188 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:11:45.021190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:11:45.021229 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:11:45.021376 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:11:45.022140 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.022994 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.023976 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.024377 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.024491 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:45.027429 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:11:45.027432 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:11:45.027433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:11:45.027443 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:11:45.029328 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:45.029823 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:45.029826 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:45.029827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:45.029930 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:11:45.030199 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:11:45.030200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:11:45.030201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:11:45.030443 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:11:45.030540 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:11:45.030825 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 20:11:45.030961 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":56,"threads_started":1}
I20260807 20:11:45.031183 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.020307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42485 (local address 127.3.15.129:42747)
0807 20:11:45.020447 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:45.020458 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:11:45.020516 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:11:45.021403 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:11:45.021406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:11:45.021809 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:11:45.022012 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:45.022017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.023105 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.023106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:45.023884 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:11:45.023887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:11:45.024482 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:11:45.024483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:11:45.024607 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:11:45.025258 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:11:45.025262 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:11:45.027024 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:11:45.029435 (+ 2411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:45.029436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:45.029437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:45.029742 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:45.030008 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:11:45.030009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:11:45.030009 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:11:45.030132 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:11:45.030520 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:11:45.030528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:11:45.030658 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 20:11:45.030769 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 20:11:45.031049 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":211,"thread_start_us":65,"threads_started":1}
W20260807 20:11:45.032078 20579 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
W20260807 20:11:45.032196 20583 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
I20260807 20:11:45.032253 20583 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94ad79cfc59148399d7c0fabd4c25d39; no voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18
I20260807 20:11:45.032491 20992 raft_consensus.cc:2726] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:11:45.627148 20997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.626902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:48045)
0807 20:11:45.627051 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:45.627082 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260807 20:11:45.682196 20998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.682006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:34087)
0807 20:11:45.682124 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:45.682158 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260807 20:11:45.807521 20999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:45.807336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:45885)
0807 20:11:45.807449 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:45.807482 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":59,"threads_started":1}
I20260807 20:11:46.627848 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:46.627633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:52495)
0807 20:11:46.627766 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:46.627800 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":68,"threads_started":1}
I20260807 20:11:46.682883 21001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:46.682666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:52989)
0807 20:11:46.682803 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:46.682842 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260807 20:11:46.808179 21002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:46.807978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:44251)
0807 20:11:46.808100 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:46.808136 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
I20260807 20:11:46.922266 21003 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:46.922398 21003 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:46.922621 21003 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
W20260807 20:11:46.923071 20583 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
W20260807 20:11:46.923156 20579 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
I20260807 20:11:46.923245 20579 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94ad79cfc59148399d7c0fabd4c25d39; no voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18
I20260807 20:11:46.923338 21003 raft_consensus.cc:2726] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:11:47.628667 21004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:47.628444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:37309)
0807 20:11:47.628581 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:47.628618 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260807 20:11:47.683772 21005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:47.683464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:60765)
0807 20:11:47.683637 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:47.683681 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260807 20:11:47.808890 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:47.808639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:47767)
0807 20:11:47.808763 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:47.808849 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
I20260807 20:11:48.629484 21010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:48.629256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:59435)
0807 20:11:48.629389 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:48.629433 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260807 20:11:48.684468 21012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:48.684252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:35909)
0807 20:11:48.684376 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:48.684426 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":60,"threads_started":1}
I20260807 20:11:48.734303 21013 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:48.734423 21013 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:48.734643 21013 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
W20260807 20:11:48.735003 20579 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
W20260807 20:11:48.735090 20583 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
I20260807 20:11:48.735163 20579 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94ad79cfc59148399d7c0fabd4c25d39; no voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18
I20260807 20:11:48.735278 21013 raft_consensus.cc:2726] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:11:48.809669 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:48.809383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:48115)
0807 20:11:48.809572 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:48.809625 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":116,"threads_started":1}
I20260807 20:11:49.630215 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:49.629967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:50595)
0807 20:11:49.630123 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:49.630169 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260807 20:11:49.685189 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:49.684965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:55065)
0807 20:11:49.685110 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:49.685144 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260807 20:11:49.810289 21017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:49.810088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:60851)
0807 20:11:49.810209 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:49.810249 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260807 20:11:50.630949 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:50.630715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:51225)
0807 20:11:50.630862 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:50.630900 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260807 20:11:50.685817 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:50.685611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:33931)
0807 20:11:50.685739 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:50.685772 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260807 20:11:50.810972 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:50.810755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:47443)
0807 20:11:50.810888 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:50.810927 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":61,"threads_started":1}
I20260807 20:11:51.006215 21022 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:51.006313 21022 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:51.006536 21022 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
W20260807 20:11:51.006953 20583 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
W20260807 20:11:51.007019 20579 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
I20260807 20:11:51.007047 20583 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94ad79cfc59148399d7c0fabd4c25d39; no voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18
I20260807 20:11:51.007109 21022 raft_consensus.cc:2726] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:11:51.631700 21023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:51.631472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:60239)
0807 20:11:51.631614 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:51.631653 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260807 20:11:51.686496 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:51.686278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:57621)
0807 20:11:51.686411 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:51.686450 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260807 20:11:51.811710 21025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:51.811485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:58713)
0807 20:11:51.811620 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:51.811664 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260807 20:11:52.632481 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:52.632245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:36949)
0807 20:11:52.632390 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:52.632433 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260807 20:11:52.687150 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:52.686942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:44887)
0807 20:11:52.687073 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:52.687105 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260807 20:11:52.812454 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:52.812190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:59567)
0807 20:11:52.812354 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:52.812406 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":63,"threads_started":1}
I20260807 20:11:53.633235 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:53.633019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:38147)
0807 20:11:53.633151 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:53.633188 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260807 20:11:53.687855 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:53.687633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:57971)
0807 20:11:53.687770 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:53.687811 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260807 20:11:53.813184 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:53.812969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:57625)
0807 20:11:53.813105 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:53.813141 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260807 20:11:54.633951 21033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:54.633721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:49565)
0807 20:11:54.633861 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:54.633901 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260807 20:11:54.688550 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:54.688326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:41647)
0807 20:11:54.688469 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:54.688506 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260807 20:11:54.813853 21035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:54.813631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:46405)
0807 20:11:54.813763 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:54.813804 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260807 20:11:55.634675 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:55.634450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:41425)
0807 20:11:55.634570 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:55.634628 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260807 20:11:55.689345 21037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:55.689143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:44145)
0807 20:11:55.689266 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:55.689302 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":62,"threads_started":1}
I20260807 20:11:55.814656 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:55.814398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:41051)
0807 20:11:55.814550 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:55.814590 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":77,"threads_started":1}
I20260807 20:11:56.635423 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:56.635168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:40503)
0807 20:11:56.635329 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:56.635373 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":66,"threads_started":1}
I20260807 20:11:56.690191 21040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:56.689927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:47731)
0807 20:11:56.690074 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:56.690128 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260807 20:11:56.815394 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:56.815164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:38047)
0807 20:11:56.815307 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:56.815347 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260807 20:11:57.636267 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:57.635973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:56347)
0807 20:11:57.636171 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:57.636217 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":90,"threads_started":1}
I20260807 20:11:57.691157 21043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:57.690953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:50721)
0807 20:11:57.691080 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:57.691112 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260807 20:11:57.816182 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:57.815933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:55705)
0807 20:11:57.816104 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:57.816137 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260807 20:11:58.199142 21045 raft_consensus.cc:482] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:11:58.199241 21045 raft_consensus.cc:504] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f47d3a4984a4ae78b77fecc6abdbb18" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42485 } } peers { permanent_uuid: "06cdb630a1174bdd951fdec8d77d1360" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 34547 } } peers { permanent_uuid: "94ad79cfc59148399d7c0fabd4c25d39" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 37155 } }
I20260807 20:11:58.199409 21045 leader_election.cc:288] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485), 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547)
W20260807 20:11:58.199795 20579 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 1f47d3a4984a4ae78b77fecc6abdbb18 (127.3.15.130:42485): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
W20260807 20:11:58.199905 20583 leader_election.cc:341] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 06cdb630a1174bdd951fdec8d77d1360 (127.3.15.131:34547): Not found: Tablet not found: fb707b0b202c4497909182f2b69befd4
I20260807 20:11:58.199983 20583 leader_election.cc:302] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 94ad79cfc59148399d7c0fabd4c25d39; no voters: 06cdb630a1174bdd951fdec8d77d1360, 1f47d3a4984a4ae78b77fecc6abdbb18
I20260807 20:11:58.200090 21045 raft_consensus.cc:2726] T fb707b0b202c4497909182f2b69befd4 P 94ad79cfc59148399d7c0fabd4c25d39 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 20:11:58.637041 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:58.636845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.129:39981)
0807 20:11:58.636965 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:58.636995 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260807 20:11:58.692018 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:58.691787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.130:38761)
0807 20:11:58.691933 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:58.691972 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260807 20:11:58.811470 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 20524
I20260807 20:11:58.816946 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:11:58.816717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:45179 (local address 127.3.15.131:37241)
0807 20:11:58.816869 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 20:11:58.816902 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.190:45179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260807 20:11:58.818293 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 20706
I20260807 20:11:58.824386 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 20848
2026-08-07T20:11:58Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58210 ms)
[----------] 30 tests from SecurityITest (236910 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:11:58 dist-test-slave-1x4g krb5kdc[21051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:11:58 dist-test-slave-1x4g krb5kdc[21051](info): set up 2 sockets
Aug 07 20:11:58 dist-test-slave-1x4g krb5kdc[21051](info): commencing operation
krb5kdc: starting...
W20260807 20:11:59.910012 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:11:59 dist-test-slave-1x4g krb5kdc[21051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:00.040875 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:37987
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:37987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:00.125077 21064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:00.125313 21064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:00.125351 21064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:00.126502 21064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:00.126569 21064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:00.126597 21064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:00.126627 21064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:00.126650 21064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:00.128144 21064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:37987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:37987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:00.128419 21064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:00.128650 21064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:00.133829 21070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:00.133888 21064 server_base.cc:1034] running on GCE node
W20260807 20:12:00.133837 21069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:00.133838 21072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:00.134479 21064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:00.134802 21064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:00.135965 21064 hybrid_clock.cc:648] HybridClock initialized: now 1786133520135950 us; error 49 us; skew 500 ppm
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:00.138579 21064 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:00.139019 21064 webserver.cc:460] Webserver started at http://127.3.15.190:34435/ using document root <none> and password file <none>
I20260807 20:12:00.139221 21064 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:00.139262 21064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:00.139353 21064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:00.140373 21064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "14c3eaddef1043e49c9e40c5b7e211fd"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "a9bedafe41afdf3c5f84ffce88f8cb0b"
server_key_iv: "32f293c30783bcd00c7e2fdc635c4cc9"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.140712 21064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "14c3eaddef1043e49c9e40c5b7e211fd"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "a9bedafe41afdf3c5f84ffce88f8cb0b"
server_key_iv: "32f293c30783bcd00c7e2fdc635c4cc9"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.142045 21064 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:00.142719 21079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.142911 21064 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:00.143003 21064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "14c3eaddef1043e49c9e40c5b7e211fd"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "a9bedafe41afdf3c5f84ffce88f8cb0b"
server_key_iv: "32f293c30783bcd00c7e2fdc635c4cc9"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.143060 21064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:00.162550 21064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:00.164721 21064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:00.164939 21064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:00.169045 21064 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:37987
I20260807 20:12:00.169159 21134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:37987 every 8 connection(s)
I20260807 20:12:00.169507 21064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:00.170508 21135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:00.173300 21135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd: Bootstrap starting.
I20260807 20:12:00.173971 21135 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:00.174290 21135 log.cc:824] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:00.175089 21135 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd: No bootstrap required, opened a new log
I20260807 20:12:00.176192 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21064
I20260807 20:12:00.176293 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:00.176524 3134 external_mini_cluster.cc:1428] Setting key 8394f0d46b85f51675aed5e4a2d2e121
I20260807 20:12:00.177129 21135 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37987 } }
I20260807 20:12:00.177369 21135 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:00.177430 21135 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c3eaddef1043e49c9e40c5b7e211fd, State: Initialized, Role: FOLLOWER
I20260807 20:12:00.177580 21135 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37987 } }
I20260807 20:12:00.177693 21135 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:00.177768 21135 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:00.177840 21135 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:00.178795 21135 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37987 } }
I20260807 20:12:00.178942 21135 leader_election.cc:302] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14c3eaddef1043e49c9e40c5b7e211fd; no voters:
I20260807 20:12:00.179140 21135 leader_election.cc:288] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:00.179171 21140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:00.179291 21140 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [term 1 LEADER]: Becoming Leader. State: Replica: 14c3eaddef1043e49c9e40c5b7e211fd, State: Running, Role: LEADER
I20260807 20:12:00.179451 21135 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:00.179450 21140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37987 } }
I20260807 20:12:00.179942 21141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37987 } } }
I20260807 20:12:00.180045 21141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:00.180522 21143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14c3eaddef1043e49c9e40c5b7e211fd. Latest consensus state: current_term: 1 leader_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c3eaddef1043e49c9e40c5b7e211fd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 37987 } } }
I20260807 20:12:00.180639 21143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:00.180866 21148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:00.181466 21148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:00.183158 21148 catalog_manager.cc:1347] Generated new cluster ID: a7e0a4e49922449d892b2951f00eba47
I20260807 20:12:00.183219 21148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133519, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:00.188728 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.177165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.3.15.190:37987)
0807 20:12:00.177356 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.177373 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.177401 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:12:00.177541 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.177549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.177612 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.177758 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:00.178444 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.179661 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.180858 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.181485 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.181601 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.184872 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:00.184876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:00.184876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:00.184900 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:00.186906 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.187343 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.187346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.187346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.187520 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.187791 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.187792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.187792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.188102 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:00.188253 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.188386 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.188526 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260807 20:12:00.194856 21148 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:00.195595 21148 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:00.204824 21148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd: Generated new TSK 0
I20260807 20:12:00.205094 21148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:12:00.242106 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37987
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:00.328015 21164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:00.328301 21164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:00.328344 21164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:00.329581 21164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:00.329659 21164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:00.329701 21164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:12:00.331207 21164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:00.331496 21164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:00.331746 21164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:00.337062 21170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:00.337069 21171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:00.337035 21173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:00.337438 21164 server_base.cc:1034] running on GCE node
I20260807 20:12:00.337702 21164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:00.337980 21164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:00.339141 21164 hybrid_clock.cc:648] HybridClock initialized: now 1786133520339109 us; error 45 us; skew 500 ppm
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:00.341921 21164 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:12:00.342319 21164 webserver.cc:460] Webserver started at http://127.3.15.129:37211/ using document root <none> and password file <none>
I20260807 20:12:00.342512 21164 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:00.342553 21164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:00.342648 21164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:00.343672 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "a59aa3abddb84650bc566c39a57ddb31"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "9f5bebd7386535106f42274bfbdddc20"
server_key_iv: "ad6793d4f5240830531e2128564fd617"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.343984 21164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "a59aa3abddb84650bc566c39a57ddb31"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "9f5bebd7386535106f42274bfbdddc20"
server_key_iv: "ad6793d4f5240830531e2128564fd617"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.345570 21164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:12:00.346406 21180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.346634 21164 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:00.346717 21164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "a59aa3abddb84650bc566c39a57ddb31"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "9f5bebd7386535106f42274bfbdddc20"
server_key_iv: "ad6793d4f5240830531e2128564fd617"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.346778 21164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:00.363925 21164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:00.366042 21164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:00.366173 21164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:00.366386 21164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:00.366731 21164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:00.366770 21164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.366816 21164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:00.366842 21164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.373257 21164 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:35191
I20260807 20:12:00.373311 21298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:35191 every 8 connection(s)
I20260807 20:12:00.373660 21164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:00.378944 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21164
I20260807 20:12:00.379062 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:00.379282 3134 external_mini_cluster.cc:1428] Setting key b571c1fd124f1f3a45680d61d1f7f60a
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:00.383988 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.374466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:45185 (local address 127.3.15.190:37987)
0807 20:12:00.374531 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.374533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.374749 (+ 216us) server_negotiation.cc:407] Connection header received
0807 20:12:00.375258 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.375259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.375273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.375329 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:00.376295 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.376832 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.377809 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.378004 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.378055 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.380641 (+ 2586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:00.380643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:00.380643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:00.380668 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:00.382532 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.382938 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.382939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.382939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.383011 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.383324 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.383325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.383325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.383452 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:00.383506 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.383850 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.383921 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:12:00.384321 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.374434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37987 (local address 127.3.15.129:45185)
0807 20:12:00.374610 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.374654 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.375174 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.375395 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.375399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.375994 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:00.376213 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.376219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.376911 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.376912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.377728 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.377731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.378077 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.378077 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.378186 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.378664 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:00.378672 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:00.380539 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:00.382623 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:00.382630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:00.382632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:00.382859 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:00.383089 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:00.383090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:00.383090 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:00.383234 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:00.383520 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:00.383528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:00.383705 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.383826 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.384015 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260807 20:12:00.384865 21299 heartbeater.cc:347] Connected to a master server at 127.3.15.190:37987
I20260807 20:12:00.384970 21299 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:00.385186 21299 heartbeater.cc:511] Master 127.3.15.190:37987 requested a full tablet report, sending...
I20260807 20:12:00.385763 21098 ts_manager.cc:194] Registered new tserver with Master: a59aa3abddb84650bc566c39a57ddb31 (127.3.15.129:35191)
I20260807 20:12:00.386409 21098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:45185
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:12:00.407779 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37987
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:00.492622 21304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:00.492888 21304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:00.492929 21304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:00.494065 21304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:00.494131 21304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:00.494174 21304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:12:00.495640 21304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:00.495918 21304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:00.496152 21304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:00.501242 21310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:00.501387 21304 server_base.cc:1034] running on GCE node
W20260807 20:12:00.501253 21311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:00.501470 21314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:00.501780 21304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:00.501998 21304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:00.503161 21304 hybrid_clock.cc:648] HybridClock initialized: now 1786133520503139 us; error 36 us; skew 500 ppm
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:00.505795 21304 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:12:00.506282 21304 webserver.cc:460] Webserver started at http://127.3.15.130:45601/ using document root <none> and password file <none>
I20260807 20:12:00.506480 21304 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:00.506521 21304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:00.506619 21304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:00.507632 21304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "8e2f1f0be10245c6a820c7cdb6d33be6"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "72c80e250f792c14cebfdb33db3e9b5f"
server_key_iv: "e708e559242a34414ee7f3a1ff705089"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.507926 21304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "8e2f1f0be10245c6a820c7cdb6d33be6"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "72c80e250f792c14cebfdb33db3e9b5f"
server_key_iv: "e708e559242a34414ee7f3a1ff705089"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.509300 21304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:00.510134 21321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.510380 21304 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:00.510483 21304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "8e2f1f0be10245c6a820c7cdb6d33be6"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "72c80e250f792c14cebfdb33db3e9b5f"
server_key_iv: "e708e559242a34414ee7f3a1ff705089"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.510545 21304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:00.525290 21304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:00.528116 21304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:00.528287 21304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:00.528594 21304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:00.529098 21304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:00.529150 21304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.529183 21304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:00.529208 21304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.535055 21304 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:46113
I20260807 20:12:00.535104 21440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:46113 every 8 connection(s)
I20260807 20:12:00.535527 21304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:00.543895 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21304
I20260807 20:12:00.543987 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:00.544234 3134 external_mini_cluster.cc:1428] Setting key 58e2240f2553063ee495f119f114b175
I20260807 20:12:00.545164 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.536312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:56241 (local address 127.3.15.190:37987)
0807 20:12:00.536384 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.536386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.536584 (+ 198us) server_negotiation.cc:407] Connection header received
0807 20:12:00.537264 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.537265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.537280 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.537339 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:00.538362 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.538911 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.539976 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.540200 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.540278 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.542421 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:00.542423 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:00.542423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:00.542431 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:00.543626 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.544079 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.544080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.544081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.544151 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.544420 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.544421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.544421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.544559 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:00.544622 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.544959 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.545049 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:12:00.545396 21444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.536284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37987 (local address 127.3.15.130:56241)
0807 20:12:00.536463 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.536500 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.537140 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.537423 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.537427 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.538035 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:00.538253 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.538259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.539016 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.539018 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.539879 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.539882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.540299 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.540300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.540434 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.540967 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:00.540975 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:00.542320 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:00.543726 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:00.543733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:00.543735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:00.544001 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:00.544232 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:00.544233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:00.544233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:00.544344 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:00.544615 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:00.544622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:00.544822 (+ 200us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.544937 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.545106 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260807 20:12:00.545876 21442 heartbeater.cc:347] Connected to a master server at 127.3.15.190:37987
I20260807 20:12:00.545992 21442 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:00.546193 21442 heartbeater.cc:511] Master 127.3.15.190:37987 requested a full tablet report, sending...
I20260807 20:12:00.546581 21098 ts_manager.cc:194] Registered new tserver with Master: 8e2f1f0be10245c6a820c7cdb6d33be6 (127.3.15.130:46113)
I20260807 20:12:00.547140 21098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:56241
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:12:00.571539 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37987
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:46583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:00.656232 21447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:00.656502 21447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:00.656545 21447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:00.657904 21447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:00.657975 21447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:00.658022 21447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:12:00.659550 21447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:46583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:37987
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:00.659829 21447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:00.660117 21447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:00.665635 21458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:00.665635 21455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:00.665658 21453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:00.665997 21447 server_base.cc:1034] running on GCE node
I20260807 20:12:00.666206 21447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:00.666465 21447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:00.667634 21447 hybrid_clock.cc:648] HybridClock initialized: now 1786133520667621 us; error 30 us; skew 500 ppm
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:00.670333 21447 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:12:00.670837 21447 webserver.cc:460] Webserver started at http://127.3.15.131:42307/ using document root <none> and password file <none>
I20260807 20:12:00.671088 21447 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:00.671157 21447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:00.671308 21447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:00.672408 21447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "34dfe374155a4b06b15d15f7735ca908"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "c99126f21c33918b15966a66d49d3bb2"
server_key_iv: "949150bde33d621b1a44063727d36f6e"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.672848 21447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "34dfe374155a4b06b15d15f7735ca908"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "c99126f21c33918b15966a66d49d3bb2"
server_key_iv: "949150bde33d621b1a44063727d36f6e"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.674206 21447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:00.674968 21466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.675129 21447 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:00.675218 21447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "34dfe374155a4b06b15d15f7735ca908"
format_stamp: "Formatted at 2026-08-07 20:12:00 on dist-test-slave-1x4g"
server_key: "c99126f21c33918b15966a66d49d3bb2"
server_key_iv: "949150bde33d621b1a44063727d36f6e"
server_key_version: "encryptionkey@0"
I20260807 20:12:00.675272 21447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:00.694459 21447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:00.696316 21447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:00.696431 21447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:00.696643 21447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:00.697006 21447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:00.697042 21447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.697065 21447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:00.697083 21447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:00.702857 21447 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:46203
I20260807 20:12:00.702919 21586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:46203 every 8 connection(s)
I20260807 20:12:00.703250 21447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:00.707214 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21447
I20260807 20:12:00.707335 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:00.707518 3134 external_mini_cluster.cc:1428] Setting key e3bb0cd83619bba13fbc404cfeb71198
Aug 07 20:12:00 dist-test-slave-1x4g krb5kdc[21051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133520, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:00.713281 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.703984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:55649 (local address 127.3.15.190:37987)
0807 20:12:00.704062 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.704064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.704227 (+ 163us) server_negotiation.cc:407] Connection header received
0807 20:12:00.704733 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.704735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.704749 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.704865 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:00.705854 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.706354 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.707513 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.707804 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.707875 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.710293 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:00.710295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:00.710295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:00.710303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:00.711788 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.712220 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.712222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.712222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.712288 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.712571 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.712572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.712573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.712698 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:00.712777 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.713098 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.713183 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:12:00.713492 21589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.703929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:37987 (local address 127.3.15.131:55649)
0807 20:12:00.704110 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.704149 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.704633 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.704955 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.704959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.705573 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:00.705770 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.705775 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.706456 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.706456 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.707268 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.707270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.707901 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.707903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.708061 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.708694 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:00.708702 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:00.710192 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:00.711893 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:00.711901 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:00.711903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:00.712140 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:00.712361 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:00.712362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:00.712362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:00.712482 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:00.712769 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:00.712806 (+ 37us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:00.712956 (+ 150us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.713073 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.713243 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260807 20:12:00.714118 21587 heartbeater.cc:347] Connected to a master server at 127.3.15.190:37987
I20260807 20:12:00.714223 21587 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:00.714376 21587 heartbeater.cc:511] Master 127.3.15.190:37987 requested a full tablet report, sending...
I20260807 20:12:00.714768 21098 ts_manager.cc:194] Registered new tserver with Master: 34dfe374155a4b06b15d15f7735ca908 (127.3.15.131:46203)
I20260807 20:12:00.715180 21098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:55649
I20260807 20:12:00.719470 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:00.726470 21139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.720808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50114 (local address 127.3.15.190:37987)
0807 20:12:00.720869 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.720871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.720975 (+ 104us) server_negotiation.cc:407] Connection header received
0807 20:12:00.721011 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.721012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.721020 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.721071 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:00.721614 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.722078 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.722797 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.722981 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.723039 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.724258 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:00.724259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:00.724259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:00.724264 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:00.725520 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.725818 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.725819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.725819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.725870 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:00.726047 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:00.726047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:00.726047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:00.726183 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:00.726256 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.726352 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.726407 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:12:00.727949 21098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:00.729664 21098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:00.736733 21598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.732049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:35191 (local address 127.0.0.1:53258)
0807 20:12:00.732330 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.732372 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.732441 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.733222 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.733223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.733225 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:00.733279 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.733280 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.734466 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.734467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.735428 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.735431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.736379 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.736379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.736480 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.736490 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.736573 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.736653 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":2,"thread_start_us":61,"threads_started":1}
I20260807 20:12:00.736792 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.732574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53258 (local address 127.3.15.129:35191)
0807 20:12:00.732991 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.732995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.733002 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:12:00.733032 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.733033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.733045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.733134 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:00.733369 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.734362 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.735527 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.736275 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.736362 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.736426 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.736635 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.736679 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":88,"threads_started":1}
I20260807 20:12:00.736801 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.732123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37384 (local address 127.3.15.131:46203)
0807 20:12:00.732279 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.732285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.732607 (+ 322us) server_negotiation.cc:407] Connection header received
0807 20:12:00.732661 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.732663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.732681 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.732754 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:00.733132 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.733990 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.735638 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.736295 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.736362 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.736426 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.736652 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.736697 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260807 20:12:00.737041 21600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.732054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46203 (local address 127.0.0.1:37384)
0807 20:12:00.732541 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.732549 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.732599 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.732843 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.732844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.732847 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:00.733048 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.733051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.734096 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.734097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.735551 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.735555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.736377 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.736379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.736474 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.736490 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.736920 (+ 430us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.736978 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":56,"thread_start_us":116,"threads_started":1}
I20260807 20:12:00.737610 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.732051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:46113 (local address 127.0.0.1:49522)
0807 20:12:00.732330 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.732372 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.732441 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.732748 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.732751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.732753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:00.732855 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.732858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.734162 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.734163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.735551 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.735555 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.737367 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.737368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.737435 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.737440 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.737507 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.737554 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":116,"thread_start_us":95,"threads_started":1}
I20260807 20:12:00.737818 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.732335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49522 (local address 127.3.15.130:46113)
0807 20:12:00.732504 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.732509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.732521 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:00.732567 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.732569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.732586 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.732675 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:00.732922 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.734064 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.736467 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.737282 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.737355 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.737433 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.737694 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.737739 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260807 20:12:00.738221 21515 tablet_service.cc:1467] Processing CreateTablet for tablet 1b14d94c498e4f56b95b05b5b3b8400b (DEFAULT_TABLE table=test-table [id=b5219956c9af463796538315aa9ff590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:00.738332 21229 tablet_service.cc:1467] Processing CreateTablet for tablet 1b14d94c498e4f56b95b05b5b3b8400b (DEFAULT_TABLE table=test-table [id=b5219956c9af463796538315aa9ff590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:00.738389 21371 tablet_service.cc:1467] Processing CreateTablet for tablet 1b14d94c498e4f56b95b05b5b3b8400b (DEFAULT_TABLE table=test-table [id=b5219956c9af463796538315aa9ff590]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:00.738636 21515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b14d94c498e4f56b95b05b5b3b8400b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:00.738690 21371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b14d94c498e4f56b95b05b5b3b8400b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:00.738698 21229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b14d94c498e4f56b95b05b5b3b8400b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:00.741516 21605 tablet_bootstrap.cc:492] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: Bootstrap starting.
I20260807 20:12:00.741990 21606 tablet_bootstrap.cc:492] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Bootstrap starting.
I20260807 20:12:00.742403 21605 tablet_bootstrap.cc:654] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:00.742659 21606 tablet_bootstrap.cc:654] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:00.743116 21605 log.cc:824] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:00.743245 21606 log.cc:824] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:00.743609 21608 tablet_bootstrap.cc:492] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Bootstrap starting.
I20260807 20:12:00.744036 21606 tablet_bootstrap.cc:492] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: No bootstrap required, opened a new log
I20260807 20:12:00.744078 21605 tablet_bootstrap.cc:492] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: No bootstrap required, opened a new log
I20260807 20:12:00.744127 21606 ts_tablet_manager.cc:1397] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:00.744138 21605 ts_tablet_manager.cc:1397] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:12:00.744282 21608 tablet_bootstrap.cc:654] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:00.744912 21608 log.cc:824] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:00.745718 21608 tablet_bootstrap.cc:492] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: No bootstrap required, opened a new log
I20260807 20:12:00.745827 21608 ts_tablet_manager.cc:1397] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:00.746091 21606 raft_consensus.cc:348] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.746259 21606 raft_consensus.cc:374] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:00.746304 21606 raft_consensus.cc:729] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2f1f0be10245c6a820c7cdb6d33be6, State: Initialized, Role: FOLLOWER
I20260807 20:12:00.746424 21606 consensus_queue.cc:260] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.746446 21605 raft_consensus.cc:348] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.746677 21605 raft_consensus.cc:374] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:00.746729 21605 raft_consensus.cc:729] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a59aa3abddb84650bc566c39a57ddb31, State: Initialized, Role: FOLLOWER
I20260807 20:12:00.746742 21606 ts_tablet_manager.cc:1428] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:00.746904 21442 heartbeater.cc:503] Master 127.3.15.190:37987 was elected leader, sending a full tablet report...
I20260807 20:12:00.747200 21605 consensus_queue.cc:260] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.747512 21605 ts_tablet_manager.cc:1428] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:00.747687 21299 heartbeater.cc:503] Master 127.3.15.190:37987 was elected leader, sending a full tablet report...
I20260807 20:12:00.748056 21608 raft_consensus.cc:348] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.748219 21608 raft_consensus.cc:374] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:00.748265 21608 raft_consensus.cc:729] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34dfe374155a4b06b15d15f7735ca908, State: Initialized, Role: FOLLOWER
I20260807 20:12:00.748378 21608 consensus_queue.cc:260] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.748680 21608 ts_tablet_manager.cc:1428] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:00.748705 21587 heartbeater.cc:503] Master 127.3.15.190:37987 was elected leader, sending a full tablet report...
W20260807 20:12:00.786160 21443 tablet.cc:2369] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:00.831421 21613 raft_consensus.cc:482] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:00.831676 21613 raft_consensus.cc:504] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.832204 21613 leader_election.cc:288] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34dfe374155a4b06b15d15f7735ca908 (127.3.15.131:46203), 8e2f1f0be10245c6a820c7cdb6d33be6 (127.3.15.130:46113)
I20260807 20:12:00.836580 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.832370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:46203 (local address 127.3.15.129:44093)
0807 20:12:00.832731 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.832743 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.832858 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.833083 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.833085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.833087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:00.833169 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.833172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.834084 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.834085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.835359 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.835362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.836277 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.836278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.836361 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.836366 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.836427 (+ 61us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.836471 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":91,"thread_start_us":207,"threads_started":1}
I20260807 20:12:00.836661 21601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.832666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:44093 (local address 127.3.15.131:46203)
0807 20:12:00.832717 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.832719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.832828 (+ 109us) server_negotiation.cc:407] Connection header received
0807 20:12:00.832965 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.832966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.832977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.833062 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:00.833249 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.833977 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.835466 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.836185 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.836280 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.836290 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.836462 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.836508 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:12:00.836826 21301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.832369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:46113 (local address 127.3.15.129:58405)
0807 20:12:00.832445 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.832458 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:00.832505 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:00.833497 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:00.833498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:00.833500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:00.833578 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.833580 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.834479 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.834480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.835720 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:00.835724 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.836514 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.836515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:00.836591 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.836595 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:00.836662 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:00.836721 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":2}
I20260807 20:12:00.837045 21603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.832490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:58405 (local address 127.3.15.130:46113)
0807 20:12:00.832537 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.832539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.832550 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:12:00.833379 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.833381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.833397 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.833483 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:00.833678 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.834384 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.835824 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.836408 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.836474 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.836481 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.836928 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.836974 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:12:00.837442 21392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b14d94c498e4f56b95b05b5b3b8400b" candidate_uuid: "a59aa3abddb84650bc566c39a57ddb31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" is_pre_election: true
I20260807 20:12:00.837638 21392 raft_consensus.cc:2454] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a59aa3abddb84650bc566c39a57ddb31 in term 0.
I20260807 20:12:00.837918 21183 leader_election.cc:302] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e2f1f0be10245c6a820c7cdb6d33be6, a59aa3abddb84650bc566c39a57ddb31; no voters:
I20260807 20:12:00.838096 21613 raft_consensus.cc:2781] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:00.838181 21613 raft_consensus.cc:482] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:00.838213 21613 raft_consensus.cc:3037] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:00.838208 21538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b14d94c498e4f56b95b05b5b3b8400b" candidate_uuid: "a59aa3abddb84650bc566c39a57ddb31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34dfe374155a4b06b15d15f7735ca908" is_pre_election: true
I20260807 20:12:00.838332 21538 raft_consensus.cc:2454] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a59aa3abddb84650bc566c39a57ddb31 in term 0.
I20260807 20:12:00.839056 21613 raft_consensus.cc:504] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.839221 21613 leader_election.cc:288] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [CANDIDATE]: Term 1 election: Requested vote from peers 34dfe374155a4b06b15d15f7735ca908 (127.3.15.131:46203), 8e2f1f0be10245c6a820c7cdb6d33be6 (127.3.15.130:46113)
I20260807 20:12:00.839406 21392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b14d94c498e4f56b95b05b5b3b8400b" candidate_uuid: "a59aa3abddb84650bc566c39a57ddb31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6"
I20260807 20:12:00.839447 21538 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b14d94c498e4f56b95b05b5b3b8400b" candidate_uuid: "a59aa3abddb84650bc566c39a57ddb31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34dfe374155a4b06b15d15f7735ca908"
I20260807 20:12:00.839500 21392 raft_consensus.cc:3037] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:00.839541 21538 raft_consensus.cc:3037] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:00.840261 21538 raft_consensus.cc:2454] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a59aa3abddb84650bc566c39a57ddb31 in term 1.
I20260807 20:12:00.840306 21392 raft_consensus.cc:2454] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a59aa3abddb84650bc566c39a57ddb31 in term 1.
I20260807 20:12:00.840471 21184 leader_election.cc:302] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34dfe374155a4b06b15d15f7735ca908, a59aa3abddb84650bc566c39a57ddb31; no voters:
I20260807 20:12:00.840577 21613 raft_consensus.cc:2781] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:00.840765 21613 raft_consensus.cc:686] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 1 LEADER]: Becoming Leader. State: Replica: a59aa3abddb84650bc566c39a57ddb31, State: Running, Role: LEADER
I20260807 20:12:00.840898 21613 consensus_queue.cc:237] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } }
I20260807 20:12:00.841940 21097 catalog_manager.cc:5495] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 reported cstate change: term changed from 0 to 1, leader changed from <none> to a59aa3abddb84650bc566c39a57ddb31 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "a59aa3abddb84650bc566c39a57ddb31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a59aa3abddb84650bc566c39a57ddb31" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 35191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:00.852046 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.849499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53262 (local address 127.3.15.129:35191)
0807 20:12:00.849568 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:00.849570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:00.849580 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:00.849634 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:00.849635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:00.849665 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:00.849753 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:00.849906 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.850450 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.851229 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:00.851435 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:00.851509 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:00.851694 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:00.851810 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:00.851910 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:00.851928 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 20:12:00.851952 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:12:00.854075 21538 raft_consensus.cc:1261] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 1 FOLLOWER]: Refusing update from remote peer a59aa3abddb84650bc566c39a57ddb31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:00.854101 21392 raft_consensus.cc:1261] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 1 FOLLOWER]: Refusing update from remote peer a59aa3abddb84650bc566c39a57ddb31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:00.854467 21613 consensus_queue.cc:1035] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e2f1f0be10245c6a820c7cdb6d33be6" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 46113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:00.854638 21617 consensus_queue.cc:1035] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34dfe374155a4b06b15d15f7735ca908" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:00.857477 21621 mvcc.cc:205] Tried to move back new op lower bound from 7316002901415055360 to 7316002901364826112. Current Snapshot: MvccSnapshot[applied={T|T < 7316002901415055360}]
I20260807 20:12:00.857959 21622 mvcc.cc:205] Tried to move back new op lower bound from 7316002901415055360 to 7316002901364826112. Current Snapshot: MvccSnapshot[applied={T|T < 7316002901415055360}]
I20260807 20:12:00.859798 21097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:00.859900 21097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:00.860754 21097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 14c3eaddef1043e49c9e40c5b7e211fd: Sending DeleteTablet for 3 replicas of tablet 1b14d94c498e4f56b95b05b5b3b8400b
I20260807 20:12:00.861136 21229 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b14d94c498e4f56b95b05b5b3b8400b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:00 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:53258
I20260807 20:12:00.861157 21371 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b14d94c498e4f56b95b05b5b3b8400b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:00 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:49522
I20260807 20:12:00.861392 21515 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b14d94c498e4f56b95b05b5b3b8400b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:00 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:37384
I20260807 20:12:00.861639 21632 tablet_replica.cc:330] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: stopping tablet replica
I20260807 20:12:00.861692 21631 tablet_replica.cc:330] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31: stopping tablet replica
I20260807 20:12:00.861748 21632 raft_consensus.cc:2229] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:00.861771 21631 raft_consensus.cc:2229] T 1b14d94c498e4f56b95b05b5b3b8400b P a59aa3abddb84650bc566c39a57ddb31 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:00.861860 21632 raft_consensus.cc:2258] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:00.861912 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21164
I20260807 20:12:00.863833 21630 tablet_replica.cc:330] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: stopping tablet replica
I20260807 20:12:00.863947 21630 raft_consensus.cc:2229] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:00.864048 21630 raft_consensus.cc:2258] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:00.865087 21632 ts_tablet_manager.cc:1905] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:00.866567 21632 ts_tablet_manager.cc:1918] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:00.866634 21632 log.cc:1196] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/1b14d94c498e4f56b95b05b5b3b8400b
I20260807 20:12:00.866930 21632 ts_tablet_manager.cc:1939] T 1b14d94c498e4f56b95b05b5b3b8400b P 34dfe374155a4b06b15d15f7735ca908: Deleting consensus metadata
I20260807 20:12:00.867388 21083 catalog_manager.cc:4841] TS 34dfe374155a4b06b15d15f7735ca908 (127.3.15.131:46203): tablet 1b14d94c498e4f56b95b05b5b3b8400b (table test-table [id=b5219956c9af463796538315aa9ff590]) successfully deleted
I20260807 20:12:00.869365 21630 ts_tablet_manager.cc:1905] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:00.870844 21630 ts_tablet_manager.cc:1918] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:00.870940 21630 log.cc:1196] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/1b14d94c498e4f56b95b05b5b3b8400b
I20260807 20:12:00.871243 21630 ts_tablet_manager.cc:1939] T 1b14d94c498e4f56b95b05b5b3b8400b P 8e2f1f0be10245c6a820c7cdb6d33be6: Deleting consensus metadata
I20260807 20:12:00.871701 21082 catalog_manager.cc:4841] TS 8e2f1f0be10245c6a820c7cdb6d33be6 (127.3.15.130:46113): tablet 1b14d94c498e4f56b95b05b5b3b8400b (table test-table [id=b5219956c9af463796538315aa9ff590]) successfully deleted
W20260807 20:12:00.874122 21080 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:35191 (error 108)
I20260807 20:12:00.874396 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21304
I20260807 20:12:00.874730 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:00.874583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:35191 (local address 127.0.0.1:53278)
0807 20:12:00.874652 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:00.874690 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:35191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 20:12:00.874835 21080 catalog_manager.cc:4584] TS a59aa3abddb84650bc566c39a57ddb31 (127.3.15.129:35191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b14d94c498e4f56b95b05b5b3b8400b: Network error: Client connection negotiation failed: client connection to 127.3.15.129:35191: connect: Connection refused (error 111)
I20260807 20:12:00.881325 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21447
I20260807 20:12:00.887909 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21064
2026-08-07T20:12:00Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2062 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T20:12:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:00Z Disabled control of system clock
I20260807 20:12:00.919010 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44289
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:44289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:01.002600 21637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:01.002838 21637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:01.002861 21637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:01.004058 21637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:01.004112 21637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:01.004125 21637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:01.004138 21637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:01.004150 21637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:01.005496 21637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:44289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:01.005702 21637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:01.005949 21637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:01.010859 21642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:01.010860 21645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:01.010848 21643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:01.011082 21637 server_base.cc:1034] running on GCE node
I20260807 20:12:01.011341 21637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:01.011665 21637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:01.012836 21637 hybrid_clock.cc:648] HybridClock initialized: now 1786133521012828 us; error 54 us; skew 500 ppm
I20260807 20:12:01.014282 21637 webserver.cc:460] Webserver started at http://127.3.15.190:39559/ using document root <none> and password file <none>
I20260807 20:12:01.014561 21637 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:01.014650 21637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:01.014807 21637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:01.015894 21637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "4bdb7698dd534a23b7a077b86e2f6998"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "9b3d5a6af77802c7ac12da24b57cbbe2"
server_key_iv: "814454718cb9377bbd1042c8eb967517"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.016294 21637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "4bdb7698dd534a23b7a077b86e2f6998"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "9b3d5a6af77802c7ac12da24b57cbbe2"
server_key_iv: "814454718cb9377bbd1042c8eb967517"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.017730 21637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:01.018465 21651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.018647 21637 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:01.018759 21637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "4bdb7698dd534a23b7a077b86e2f6998"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "9b3d5a6af77802c7ac12da24b57cbbe2"
server_key_iv: "814454718cb9377bbd1042c8eb967517"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.018833 21637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:01.029605 21637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:01.029937 21637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:01.030092 21637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:01.034219 21637 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:44289
I20260807 20:12:01.034245 21706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:44289 every 8 connection(s)
I20260807 20:12:01.034624 21637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:01.035615 21707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:01.038296 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998: Bootstrap starting.
I20260807 20:12:01.038915 21707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:01.039242 21707 log.cc:824] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:01.039947 21707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998: No bootstrap required, opened a new log
I20260807 20:12:01.041692 21707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bdb7698dd534a23b7a077b86e2f6998" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44289 } }
I20260807 20:12:01.041834 21707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:01.041857 21707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bdb7698dd534a23b7a077b86e2f6998, State: Initialized, Role: FOLLOWER
I20260807 20:12:01.042039 21707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bdb7698dd534a23b7a077b86e2f6998" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44289 } }
I20260807 20:12:01.042106 21707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:01.042150 21707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:01.042201 21707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:01.042749 21707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bdb7698dd534a23b7a077b86e2f6998" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44289 } }
I20260807 20:12:01.042867 21707 leader_election.cc:302] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bdb7698dd534a23b7a077b86e2f6998; no voters:
I20260807 20:12:01.043066 21707 leader_election.cc:288] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:01.043139 21711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:01.043322 21711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [term 1 LEADER]: Becoming Leader. State: Replica: 4bdb7698dd534a23b7a077b86e2f6998, State: Running, Role: LEADER
I20260807 20:12:01.043350 21707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:01.043427 21711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bdb7698dd534a23b7a077b86e2f6998" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44289 } }
I20260807 20:12:01.043839 21713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bdb7698dd534a23b7a077b86e2f6998. Latest consensus state: current_term: 1 leader_uuid: "4bdb7698dd534a23b7a077b86e2f6998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bdb7698dd534a23b7a077b86e2f6998" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44289 } } }
I20260807 20:12:01.043918 21712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bdb7698dd534a23b7a077b86e2f6998" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bdb7698dd534a23b7a077b86e2f6998" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44289 } } }
I20260807 20:12:01.043941 21713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:01.043967 21712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:01.044042 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21637
I20260807 20:12:01.044164 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:01.044169 21720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:01.044358 3134 external_mini_cluster.cc:1428] Setting key b1177040dd5228ed8638f00e9f5691c8
I20260807 20:12:01.044656 21720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:01.046857 21720 catalog_manager.cc:1347] Generated new cluster ID: 7073a09dd5b348fa948130baf8e93241
I20260807 20:12:01.046916 21720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:01.050700 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.045012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.3.15.190:44289)
0807 20:12:01.045473 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.045482 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.045996 (+ 514us) server_negotiation.cc:407] Connection header received
0807 20:12:01.046171 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.046178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.046252 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.046400 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:01.046650 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.047737 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.048562 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.049099 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.049246 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.049991 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:01.049995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:01.049995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:01.050188 (+ 193us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:01.050230 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:01.050242 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:01.050323 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.050401 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.050483 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":79,"threads_started":1}
I20260807 20:12:01.072414 21720 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:01.073918 21720 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:01.079983 21720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998: Generated new TSK 0
I20260807 20:12:01.080236 21720 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:01.090183 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44289
--builtin_ntp_servers=127.3.15.148:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:01.186761 21732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:01.186959 21732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:01.186981 21732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:01.188141 21732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:01.188184 21732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:01.188208 21732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:12:01.189584 21732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:01.189827 21732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:01.190157 21732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:01.195492 21740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:01.195477 21739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:01.195655 21732 server_base.cc:1034] running on GCE node
W20260807 20:12:01.195477 21742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:01.195994 21732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:01.196275 21732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:01.197440 21732 hybrid_clock.cc:648] HybridClock initialized: now 1786133521197419 us; error 56 us; skew 500 ppm
I20260807 20:12:01.198726 21732 webserver.cc:460] Webserver started at http://127.3.15.129:38761/ using document root <none> and password file <none>
I20260807 20:12:01.198993 21732 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:01.199066 21732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:01.199232 21732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:01.200325 21732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "7c1a2393b52a4118869b5ed76587c5df"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "3a1a3c3514d1419808a52d08fa5facae"
server_key_iv: "86597d434359b810ff37d7979df399c3"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.200697 21732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "7c1a2393b52a4118869b5ed76587c5df"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "3a1a3c3514d1419808a52d08fa5facae"
server_key_iv: "86597d434359b810ff37d7979df399c3"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.202035 21732 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:01.202817 21748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.203001 21732 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:01.203117 21732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "7c1a2393b52a4118869b5ed76587c5df"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "3a1a3c3514d1419808a52d08fa5facae"
server_key_iv: "86597d434359b810ff37d7979df399c3"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.203207 21732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:01.229220 21732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:01.229590 21732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:01.229761 21732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:01.230054 21732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:01.230434 21732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:01.230492 21732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.230536 21732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:01.230582 21732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.236514 21732 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:36847
I20260807 20:12:01.236663 21869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:36847 every 8 connection(s)
I20260807 20:12:01.237021 21732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:01.237293 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21732
I20260807 20:12:01.237419 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:01.237671 3134 external_mini_cluster.cc:1428] Setting key 1030161f3efb6bb2228f0722d0758684
I20260807 20:12:01.238903 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44289
--builtin_ntp_servers=127.3.15.148:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:12:01.243691 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.237878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:40913 (local address 127.3.15.190:44289)
0807 20:12:01.237974 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.238122 (+ 148us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.238313 (+ 191us) server_negotiation.cc:407] Connection header received
0807 20:12:01.239084 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.239086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.239103 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.239134 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:01.240235 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.240960 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.242018 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.242241 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.242324 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.243359 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:01.243360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:01.243361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:01.243368 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:01.243385 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:01.243390 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:01.243458 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.243577 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.243610 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:12:01.243923 21872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.237883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44289 (local address 127.3.15.129:40913)
0807 20:12:01.238158 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.238220 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.238943 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.239337 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.239342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.239353 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:01.240124 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.240132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.241062 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.241063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.241903 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.241906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.242336 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.242337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.242492 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.243130 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:01.243130 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:01.243184 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:01.243218 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:01.243471 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:01.243485 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:01.243488 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.243579 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.243688 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":101,"threads_started":1}
I20260807 20:12:01.244449 21870 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44289
I20260807 20:12:01.244570 21870 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:01.244825 21870 heartbeater.cc:511] Master 127.3.15.190:44289 requested a full tablet report, sending...
I20260807 20:12:01.245481 21668 ts_manager.cc:194] Registered new tserver with Master: 7c1a2393b52a4118869b5ed76587c5df (127.3.15.129:36847)
I20260807 20:12:01.246134 21668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.129:40913
W20260807 20:12:01.323468 21873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:01.323673 21873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:01.323694 21873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:01.324882 21873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:01.324928 21873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:01.324954 21873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:12:01.326313 21873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:01.326563 21873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:01.326869 21873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:01.331926 21879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:01.331979 21881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:01.331961 21878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:01.332118 21873 server_base.cc:1034] running on GCE node
I20260807 20:12:01.332316 21873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:01.332573 21873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:01.333745 21873 hybrid_clock.cc:648] HybridClock initialized: now 1786133521333708 us; error 51 us; skew 500 ppm
I20260807 20:12:01.334995 21873 webserver.cc:460] Webserver started at http://127.3.15.130:38221/ using document root <none> and password file <none>
I20260807 20:12:01.335232 21873 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:01.335299 21873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:01.335464 21873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:01.336531 21873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "971f772ea1e349d59d03c7c0c7948aaa"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "14e6f8a055735074b72883859fd42df8"
server_key_iv: "7abb3ff2ff2771144da61106c0fb0d26"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.336951 21873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "971f772ea1e349d59d03c7c0c7948aaa"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "14e6f8a055735074b72883859fd42df8"
server_key_iv: "7abb3ff2ff2771144da61106c0fb0d26"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.338374 21873 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:01.339150 21888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.339320 21873 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:01.339422 21873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "971f772ea1e349d59d03c7c0c7948aaa"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "14e6f8a055735074b72883859fd42df8"
server_key_iv: "7abb3ff2ff2771144da61106c0fb0d26"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.339506 21873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:01.350826 21873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:01.351163 21873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:01.351342 21873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:01.351595 21873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:01.351970 21873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:01.352030 21873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.352072 21873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:01.352101 21873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.358724 21873 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:36379
I20260807 20:12:01.358783 22008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:36379 every 8 connection(s)
I20260807 20:12:01.359141 21873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:01.364315 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.359866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:42437 (local address 127.3.15.190:44289)
0807 20:12:01.359953 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.359955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.360229 (+ 274us) server_negotiation.cc:407] Connection header received
0807 20:12:01.360744 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.360746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.360756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.360831 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:01.361575 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.362065 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.363073 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.363267 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.363323 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.364100 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:01.364101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:01.364102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:01.364106 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:01.364122 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:01.364126 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:01.364169 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.364229 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.364254 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 20:12:01.364522 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.359835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44289 (local address 127.3.15.130:42437)
0807 20:12:01.360099 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.360136 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.360651 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.360918 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.360922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.360931 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:01.361494 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.361499 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.362175 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.362177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.362970 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.362972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.363346 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.363347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.363460 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.363949 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:01.363949 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:01.363977 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:01.364000 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:01.364169 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:01.364171 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:01.364174 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.364221 (+ 47us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.364300 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":137,"threads_started":1}
I20260807 20:12:01.365080 22009 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44289
I20260807 20:12:01.365103 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 21873
I20260807 20:12:01.365185 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:01.365200 22009 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:01.365402 22009 heartbeater.cc:511] Master 127.3.15.190:44289 requested a full tablet report, sending...
I20260807 20:12:01.365414 3134 external_mini_cluster.cc:1428] Setting key 3eccd28a7f597a5e9d02a9afb5fe07d2
I20260807 20:12:01.365840 21668 ts_manager.cc:194] Registered new tserver with Master: 971f772ea1e349d59d03c7c0c7948aaa (127.3.15.130:36379)
I20260807 20:12:01.366354 21668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.130:42437
I20260807 20:12:01.366638 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44289
--builtin_ntp_servers=127.3.15.148:36145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:01.451844 22012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:01.452092 22012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:01.452128 22012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:01.453474 22012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:01.453523 22012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:01.453558 22012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:12:01.455195 22012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:36145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:01.455443 22012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:01.455760 22012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:01.460863 22017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:01.460870 22020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:01.460914 22012 server_base.cc:1034] running on GCE node
W20260807 20:12:01.460913 22018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:01.461395 22012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:01.461655 22012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:01.462823 22012 hybrid_clock.cc:648] HybridClock initialized: now 1786133521462808 us; error 45 us; skew 500 ppm
I20260807 20:12:01.464056 22012 webserver.cc:460] Webserver started at http://127.3.15.131:41793/ using document root <none> and password file <none>
I20260807 20:12:01.464309 22012 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:01.464376 22012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:01.464534 22012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:01.465715 22012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "88f19db79313411d9dfb03ba9b27e2e0"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "5d2605841fcb8ab7607db01c43c00420"
server_key_iv: "2b512302202ee1d72c6b6173e155fa6d"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.466091 22012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "88f19db79313411d9dfb03ba9b27e2e0"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "5d2605841fcb8ab7607db01c43c00420"
server_key_iv: "2b512302202ee1d72c6b6173e155fa6d"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.467413 22012 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:12:01.468129 22026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.468304 22012 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:01.468405 22012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "88f19db79313411d9dfb03ba9b27e2e0"
format_stamp: "Formatted at 2026-08-07 20:12:01 on dist-test-slave-1x4g"
server_key: "5d2605841fcb8ab7607db01c43c00420"
server_key_iv: "2b512302202ee1d72c6b6173e155fa6d"
server_key_version: "encryptionkey@0"
I20260807 20:12:01.468487 22012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:01.477654 22012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:01.477953 22012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:01.478102 22012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:01.478322 22012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:01.478682 22012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:01.478739 22012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.478781 22012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:01.478809 22012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:01.484460 22012 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:45609
I20260807 20:12:01.484499 22146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:45609 every 8 connection(s)
I20260807 20:12:01.484900 22012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:01.489964 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.485685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:60441 (local address 127.3.15.190:44289)
0807 20:12:01.485751 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.485753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.486010 (+ 257us) server_negotiation.cc:407] Connection header received
0807 20:12:01.486497 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.486498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.486507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.486577 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:01.487234 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.487763 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.488725 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.488929 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.488980 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.489745 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:01.489746 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:01.489746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:01.489751 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:01.489766 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:01.489770 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:01.489816 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.489882 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.489906 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:12:01.490151 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.485664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44289 (local address 127.3.15.131:60441)
0807 20:12:01.485868 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.485918 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.486412 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.486636 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.486639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.486647 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:01.487158 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.487162 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.487836 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.487837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.488644 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.488647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.489006 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.489007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.489137 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.489613 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:01.489613 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:01.489640 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:01.489664 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:01.489819 (+ 155us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:01.489822 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:01.489824 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.489870 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.489944 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260807 20:12:01.490656 22147 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44289
I20260807 20:12:01.490751 22147 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:01.490893 22147 heartbeater.cc:511] Master 127.3.15.190:44289 requested a full tablet report, sending...
I20260807 20:12:01.491251 21668 ts_manager.cc:194] Registered new tserver with Master: 88f19db79313411d9dfb03ba9b27e2e0 (127.3.15.131:45609)
I20260807 20:12:01.491576 21668 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.131:60441
I20260807 20:12:01.492375 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22012
I20260807 20:12:01.492457 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:01.492623 3134 external_mini_cluster.cc:1428] Setting key 770c2fae35e1a09d4a579a3669ea2e0a
I20260807 20:12:01.493579 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:01.497648 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.494917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45588 (local address 127.3.15.190:44289)
0807 20:12:01.494980 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.494982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.495140 (+ 158us) server_negotiation.cc:407] Connection header received
0807 20:12:01.495176 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.495177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.495187 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.495232 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:01.495389 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.495833 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.496557 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.496746 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.496864 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.497424 (+ 560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:01.497425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:01.497425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:01.497428 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:01.497441 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:01.497445 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:01.497487 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.497530 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.497550 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:12:01.499188 21668 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:01.500744 21668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:01.507396 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.503082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.3.15.130:36379)
0807 20:12:01.503303 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.503308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.503440 (+ 132us) server_negotiation.cc:407] Connection header received
0807 20:12:01.503590 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.503593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.503612 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.503692 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:01.503945 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.505151 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.506385 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.507004 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.507091 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.507246 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.507276 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.507313 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
I20260807 20:12:01.507383 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.502979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:36379 (local address 127.0.0.1:49270)
0807 20:12:01.503315 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.503359 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.503442 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.503789 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.503792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.503795 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:01.503856 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.503858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.505237 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.505238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.506295 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.506299 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.507094 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.507095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.507172 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.507185 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.507248 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.507304 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":29,"thread_start_us":58,"threads_started":1}
I20260807 20:12:01.507859 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.503204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:36847 (local address 127.0.0.1:43012)
0807 20:12:01.503587 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.503608 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.503656 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.503872 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.503873 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.503875 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:01.503921 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.503923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.505190 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.505191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.506548 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.506552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.507585 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.507586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.507663 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.507668 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.507726 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.507774 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":38,"thread_start_us":69,"threads_started":1}
I20260807 20:12:01.508033 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.503313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.3.15.129:36847)
0807 20:12:01.503534 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.503539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.503714 (+ 175us) server_negotiation.cc:407] Connection header received
0807 20:12:01.503761 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.503763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.503778 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.503876 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:01.503993 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.505108 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.506659 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.507391 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.507464 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.507549 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.507818 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.507866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
I20260807 20:12:01.508244 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.503340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51310 (local address 127.3.15.131:45609)
0807 20:12:01.503997 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.504001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.504010 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:01.504042 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.504043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.504064 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.504149 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:01.504253 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.505009 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.506671 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.507391 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.507647 (+ 256us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.507741 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.508117 (+ 376us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.508166 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":109,"threads_started":1}
I20260807 20:12:01.508276 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.502979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:45609 (local address 127.0.0.1:51310)
0807 20:12:01.503575 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.503585 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.503630 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.504148 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.504149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.504151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:01.504197 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.504199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.505101 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.505102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.506583 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.506585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.507776 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.507777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.507903 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.507908 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.508161 (+ 253us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.508202 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":71,"spinlock_wait_cycles":768,"thread_start_us":103,"threads_started":1}
I20260807 20:12:01.508322 21941 tablet_service.cc:1467] Processing CreateTablet for tablet 82b647f8595e4bbc9bcf936455558228 (DEFAULT_TABLE table=test-table [id=bb8d94b042e2467590f273c69a110a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:01.508723 21941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b647f8595e4bbc9bcf936455558228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:01.509423 22077 tablet_service.cc:1467] Processing CreateTablet for tablet 82b647f8595e4bbc9bcf936455558228 (DEFAULT_TABLE table=test-table [id=bb8d94b042e2467590f273c69a110a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:01.509788 22077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b647f8595e4bbc9bcf936455558228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:01.509752 21798 tablet_service.cc:1467] Processing CreateTablet for tablet 82b647f8595e4bbc9bcf936455558228 (DEFAULT_TABLE table=test-table [id=bb8d94b042e2467590f273c69a110a72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:01.510005 21798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82b647f8595e4bbc9bcf936455558228. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:01.512414 22161 tablet_bootstrap.cc:492] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Bootstrap starting.
I20260807 20:12:01.512815 22162 tablet_bootstrap.cc:492] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: Bootstrap starting.
I20260807 20:12:01.513299 22161 tablet_bootstrap.cc:654] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:01.513517 22162 tablet_bootstrap.cc:654] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:01.513672 22161 log.cc:824] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:01.513823 22162 log.cc:824] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:01.514179 22163 tablet_bootstrap.cc:492] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Bootstrap starting.
I20260807 20:12:01.514480 22161 tablet_bootstrap.cc:492] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: No bootstrap required, opened a new log
I20260807 20:12:01.514546 22162 tablet_bootstrap.cc:492] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: No bootstrap required, opened a new log
I20260807 20:12:01.514569 22161 ts_tablet_manager.cc:1397] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:12:01.514604 22162 ts_tablet_manager.cc:1397] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:01.516234 22163 tablet_bootstrap.cc:654] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:01.516564 22163 log.cc:824] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:01.516623 22161 raft_consensus.cc:348] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.516876 22161 raft_consensus.cc:374] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:01.516924 22161 raft_consensus.cc:729] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 971f772ea1e349d59d03c7c0c7948aaa, State: Initialized, Role: FOLLOWER
I20260807 20:12:01.517055 22161 consensus_queue.cc:260] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.517061 22162 raft_consensus.cc:348] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.517215 22162 raft_consensus.cc:374] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:01.517238 22162 raft_consensus.cc:729] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c1a2393b52a4118869b5ed76587c5df, State: Initialized, Role: FOLLOWER
I20260807 20:12:01.517364 22161 ts_tablet_manager.cc:1428] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 20:12:01.517376 22162 consensus_queue.cc:260] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.517495 22009 heartbeater.cc:503] Master 127.3.15.190:44289 was elected leader, sending a full tablet report...
I20260807 20:12:01.517647 22162 ts_tablet_manager.cc:1428] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:01.517680 22163 tablet_bootstrap.cc:492] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: No bootstrap required, opened a new log
I20260807 20:12:01.517735 22163 ts_tablet_manager.cc:1397] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 20:12:01.519740 22163 raft_consensus.cc:348] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.519928 22163 raft_consensus.cc:374] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:01.519973 22163 raft_consensus.cc:729] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88f19db79313411d9dfb03ba9b27e2e0, State: Initialized, Role: FOLLOWER
I20260807 20:12:01.520092 22163 consensus_queue.cc:260] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.520378 22163 ts_tablet_manager.cc:1428] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:12:01.520416 22147 heartbeater.cc:503] Master 127.3.15.190:44289 was elected leader, sending a full tablet report...
I20260807 20:12:01.520525 21870 heartbeater.cc:503] Master 127.3.15.190:44289 was elected leader, sending a full tablet report...
I20260807 20:12:01.529441 22167 raft_consensus.cc:482] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:01.529541 22167 raft_consensus.cc:504] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.529950 22167 leader_election.cc:288] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88f19db79313411d9dfb03ba9b27e2e0 (127.3.15.131:45609), 7c1a2393b52a4118869b5ed76587c5df (127.3.15.129:36847)
I20260807 20:12:01.533612 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.530094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:45609 (local address 127.3.15.130:45971)
0807 20:12:01.530185 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.530200 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.530246 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.530415 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.530417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.530419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:01.530499 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.530501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.531479 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.531480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.532820 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.532822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.533401 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.533402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.533461 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.533465 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.533511 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.533547 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"mutex_wait_us":8}
I20260807 20:12:01.533665 22160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.530219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:45971 (local address 127.3.15.131:45609)
0807 20:12:01.530279 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.530281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.530291 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:01.530320 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.530321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.530330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.530385 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:01.530609 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.531390 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.532899 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.533324 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.533387 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.533394 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.533542 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.533594 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:12:01.533990 22098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b647f8595e4bbc9bcf936455558228" candidate_uuid: "971f772ea1e349d59d03c7c0c7948aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f19db79313411d9dfb03ba9b27e2e0" is_pre_election: true
I20260807 20:12:01.534201 22098 raft_consensus.cc:2454] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 971f772ea1e349d59d03c7c0c7948aaa in term 0.
I20260807 20:12:01.534323 22170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.530112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:36847 (local address 127.3.15.130:37925)
0807 20:12:01.530472 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.530481 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:01.530520 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:01.530667 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:01.530668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:01.530669 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:01.530729 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.530730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.531544 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.531545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.532819 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:01.532822 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.534062 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.534063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:01.534136 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.534139 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:01.534189 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:12:01.534228 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":36,"thread_start_us":109,"threads_started":1}
I20260807 20:12:01.534361 22159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.530339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:37925 (local address 127.3.15.129:36847)
0807 20:12:01.530417 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.530419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.530532 (+ 113us) server_negotiation.cc:407] Connection header received
0807 20:12:01.530583 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.530584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.530598 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.530657 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:01.530795 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.531481 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.533431 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.533963 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.534053 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.534070 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.534229 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.534279 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 20:12:01.534551 21889 leader_election.cc:302] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88f19db79313411d9dfb03ba9b27e2e0, 971f772ea1e349d59d03c7c0c7948aaa; no voters:
I20260807 20:12:01.534644 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b647f8595e4bbc9bcf936455558228" candidate_uuid: "971f772ea1e349d59d03c7c0c7948aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1a2393b52a4118869b5ed76587c5df" is_pre_election: true
I20260807 20:12:01.534684 22167 raft_consensus.cc:2781] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:01.534735 22167 raft_consensus.cc:482] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:01.534755 22167 raft_consensus.cc:3037] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:01.534804 21820 raft_consensus.cc:2454] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 971f772ea1e349d59d03c7c0c7948aaa in term 0.
I20260807 20:12:01.535593 22167 raft_consensus.cc:504] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.535742 22167 leader_election.cc:288] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [CANDIDATE]: Term 1 election: Requested vote from peers 88f19db79313411d9dfb03ba9b27e2e0 (127.3.15.131:45609), 7c1a2393b52a4118869b5ed76587c5df (127.3.15.129:36847)
I20260807 20:12:01.535910 22098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b647f8595e4bbc9bcf936455558228" candidate_uuid: "971f772ea1e349d59d03c7c0c7948aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88f19db79313411d9dfb03ba9b27e2e0"
I20260807 20:12:01.535933 21820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "82b647f8595e4bbc9bcf936455558228" candidate_uuid: "971f772ea1e349d59d03c7c0c7948aaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c1a2393b52a4118869b5ed76587c5df"
I20260807 20:12:01.536000 21820 raft_consensus.cc:3037] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:01.536000 22098 raft_consensus.cc:3037] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:01.536998 21820 raft_consensus.cc:2454] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 971f772ea1e349d59d03c7c0c7948aaa in term 1.
I20260807 20:12:01.536998 22098 raft_consensus.cc:2454] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 971f772ea1e349d59d03c7c0c7948aaa in term 1.
I20260807 20:12:01.537262 21889 leader_election.cc:302] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88f19db79313411d9dfb03ba9b27e2e0, 971f772ea1e349d59d03c7c0c7948aaa; no voters:
I20260807 20:12:01.537400 22167 raft_consensus.cc:2781] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:01.537616 22167 raft_consensus.cc:686] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 1 LEADER]: Becoming Leader. State: Replica: 971f772ea1e349d59d03c7c0c7948aaa, State: Running, Role: LEADER
I20260807 20:12:01.537736 22167 consensus_queue.cc:237] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } }
I20260807 20:12:01.538698 21668 catalog_manager.cc:5495] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 971f772ea1e349d59d03c7c0c7948aaa (127.3.15.130). New cstate: current_term: 1 leader_uuid: "971f772ea1e349d59d03c7c0c7948aaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "971f772ea1e349d59d03c7c0c7948aaa" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 36379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:01.544180 22157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.541387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49286 (local address 127.3.15.130:36379)
0807 20:12:01.541434 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:01.541436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:01.541445 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:01.541524 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:01.541524 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:01.541543 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:01.541628 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:01.541731 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.542280 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.543184 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:01.543389 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:01.543473 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:01.543678 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:01.543780 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:01.543899 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:01.544007 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:12:01.544029 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:01.546269 21820 raft_consensus.cc:1261] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 1 FOLLOWER]: Refusing update from remote peer 971f772ea1e349d59d03c7c0c7948aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:01.546347 22098 raft_consensus.cc:1261] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 1 FOLLOWER]: Refusing update from remote peer 971f772ea1e349d59d03c7c0c7948aaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:01.546638 22171 consensus_queue.cc:1035] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c1a2393b52a4118869b5ed76587c5df" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 36847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:01.546773 22167 consensus_queue.cc:1035] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "88f19db79313411d9dfb03ba9b27e2e0" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 45609 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:01.550132 22176 mvcc.cc:205] Tried to move back new op lower bound from 7316002904250318848 to 7316002904219258880. Current Snapshot: MvccSnapshot[applied={T|T < 7316002904250318848}]
I20260807 20:12:01.551388 22177 mvcc.cc:205] Tried to move back new op lower bound from 7316002904250318848 to 7316002904219258880. Current Snapshot: MvccSnapshot[applied={T|T < 7316002904250318848 or (T in {7316002904250318848})}]
I20260807 20:12:01.551708 21667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:01.551791 21667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:01.552573 21667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bdb7698dd534a23b7a077b86e2f6998: Sending DeleteTablet for 3 replicas of tablet 82b647f8595e4bbc9bcf936455558228
I20260807 20:12:01.552918 22077 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b647f8595e4bbc9bcf936455558228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:01 UTC) from {username='slave'} at 127.0.0.1:51310
I20260807 20:12:01.552963 21798 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b647f8595e4bbc9bcf936455558228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:01 UTC) from {username='slave'} at 127.0.0.1:43012
I20260807 20:12:01.553193 22186 tablet_replica.cc:330] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df: stopping tablet replica
I20260807 20:12:01.553226 21941 tablet_service.cc:1514] Processing DeleteTablet for tablet 82b647f8595e4bbc9bcf936455558228 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:01 UTC) from {username='slave'} at 127.0.0.1:49270
I20260807 20:12:01.553299 22186 raft_consensus.cc:2229] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:01.553340 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21732
I20260807 20:12:01.553402 22186 raft_consensus.cc:2258] T 82b647f8595e4bbc9bcf936455558228 P 7c1a2393b52a4118869b5ed76587c5df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:01.553478 22187 tablet_replica.cc:330] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: stopping tablet replica
I20260807 20:12:01.553558 22187 raft_consensus.cc:2229] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:01.553647 22187 raft_consensus.cc:2258] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:01.555101 22187 ts_tablet_manager.cc:1905] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:01.556954 22187 ts_tablet_manager.cc:1918] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:01.557027 22187 log.cc:1196] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/82b647f8595e4bbc9bcf936455558228
I20260807 20:12:01.557296 22187 ts_tablet_manager.cc:1939] T 82b647f8595e4bbc9bcf936455558228 P 971f772ea1e349d59d03c7c0c7948aaa: Deleting consensus metadata
I20260807 20:12:01.558105 21655 catalog_manager.cc:4841] TS 971f772ea1e349d59d03c7c0c7948aaa (127.3.15.130:36379): tablet 82b647f8595e4bbc9bcf936455558228 (table test-table [id=bb8d94b042e2467590f273c69a110a72]) successfully deleted
W20260807 20:12:01.561947 21655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:36847 (error 108)
I20260807 20:12:01.561985 22185 tablet_replica.cc:330] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: stopping tablet replica
I20260807 20:12:01.562175 22185 raft_consensus.cc:2229] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:01.562317 22185 raft_consensus.cc:2258] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:01.562403 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21873
I20260807 20:12:01.564761 22185 ts_tablet_manager.cc:1905] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:01.565900 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:01.565754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:36847 (local address 127.0.0.1:43022)
0807 20:12:01.565814 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:01.565870 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:36847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 20:12:01.566053 21655 catalog_manager.cc:4584] TS 7c1a2393b52a4118869b5ed76587c5df (127.3.15.129:36847): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 82b647f8595e4bbc9bcf936455558228: Network error: Client connection negotiation failed: client connection to 127.3.15.129:36847: connect: Connection refused (error 111)
I20260807 20:12:01.566923 22185 ts_tablet_manager.cc:1918] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:01.566988 22185 log.cc:1196] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/82b647f8595e4bbc9bcf936455558228
I20260807 20:12:01.567282 22185 ts_tablet_manager.cc:1939] T 82b647f8595e4bbc9bcf936455558228 P 88f19db79313411d9dfb03ba9b27e2e0: Deleting consensus metadata
I20260807 20:12:01.567725 21652 catalog_manager.cc:4841] TS 88f19db79313411d9dfb03ba9b27e2e0 (127.3.15.131:45609): tablet 82b647f8595e4bbc9bcf936455558228 (table test-table [id=bb8d94b042e2467590f273c69a110a72]) successfully deleted
I20260807 20:12:01.570259 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22012
I20260807 20:12:01.577155 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 21637
2026-08-07T20:12:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (688 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:01 dist-test-slave-1x4g krb5kdc[22191](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:01 dist-test-slave-1x4g krb5kdc[22191](info): set up 2 sockets
Aug 07 20:12:01 dist-test-slave-1x4g krb5kdc[22191](info): commencing operation
krb5kdc: starting...
W20260807 20:12:02.645817 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:02 dist-test-slave-1x4g krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:02.771833 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:40365
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:40365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:02.863480 22204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:02.863718 22204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:02.863755 22204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:02.864910 22204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:02.864982 22204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:02.865010 22204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:02.865041 22204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:02.865067 22204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:02.866531 22204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:40365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:40365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:02.866798 22204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:02.867072 22204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:02.871876 22210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:02.871860 22211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:02.871860 22213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:02.872112 22204 server_base.cc:1034] running on GCE node
I20260807 20:12:02.872363 22204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:02.872694 22204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:02.873854 22204 hybrid_clock.cc:648] HybridClock initialized: now 1786133522873830 us; error 43 us; skew 500 ppm
Aug 07 20:12:02 dist-test-slave-1x4g krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133522, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:02.876389 22204 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:02.876900 22204 webserver.cc:460] Webserver started at http://127.3.15.190:41453/ using document root <none> and password file <none>
I20260807 20:12:02.877146 22204 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:02.877216 22204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:02.877368 22204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:02.878408 22204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "ecf68924cf1d44d4bfedec9f73009950"
format_stamp: "Formatted at 2026-08-07 20:12:02 on dist-test-slave-1x4g"
server_key: "c189c3945570c27823c7066ecea36095"
server_key_iv: "eb14e40d80abb3d98be7b4ea191d591a"
server_key_version: "encryptionkey@0"
I20260807 20:12:02.878791 22204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "ecf68924cf1d44d4bfedec9f73009950"
format_stamp: "Formatted at 2026-08-07 20:12:02 on dist-test-slave-1x4g"
server_key: "c189c3945570c27823c7066ecea36095"
server_key_iv: "eb14e40d80abb3d98be7b4ea191d591a"
server_key_version: "encryptionkey@0"
I20260807 20:12:02.880438 22204 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:02.881299 22221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:02.881486 22204 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:02.881593 22204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "ecf68924cf1d44d4bfedec9f73009950"
format_stamp: "Formatted at 2026-08-07 20:12:02 on dist-test-slave-1x4g"
server_key: "c189c3945570c27823c7066ecea36095"
server_key_iv: "eb14e40d80abb3d98be7b4ea191d591a"
server_key_version: "encryptionkey@0"
I20260807 20:12:02.881665 22204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:02.890641 22204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:02.892418 22204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:02.892586 22204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:02.896464 22204 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:40365
I20260807 20:12:02.896526 22276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:40365 every 8 connection(s)
I20260807 20:12:02.896938 22204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:02.897432 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22204
I20260807 20:12:02.897560 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:02.897749 3134 external_mini_cluster.cc:1428] Setting key eba3e9be7f5ae85209ed2c44e4894abf
I20260807 20:12:02.897966 22278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:02.901427 22278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950: Bootstrap starting.
I20260807 20:12:02.902047 22278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:02.902390 22278 log.cc:824] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:02.903170 22278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950: No bootstrap required, opened a new log
Aug 07 20:12:02 dist-test-slave-1x4g krb5kdc[22191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:02.905208 22278 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf68924cf1d44d4bfedec9f73009950" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 40365 } }
I20260807 20:12:02.905393 22278 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:02.905437 22278 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf68924cf1d44d4bfedec9f73009950, State: Initialized, Role: FOLLOWER
I20260807 20:12:02.905583 22278 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf68924cf1d44d4bfedec9f73009950" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 40365 } }
I20260807 20:12:02.905689 22278 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:02.905730 22278 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:02.905787 22278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:02.906477 22278 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf68924cf1d44d4bfedec9f73009950" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 40365 } }
I20260807 20:12:02.906567 22278 leader_election.cc:302] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecf68924cf1d44d4bfedec9f73009950; no voters:
I20260807 20:12:02.906762 22278 leader_election.cc:288] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:02.906800 22283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:02.906930 22283 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [term 1 LEADER]: Becoming Leader. State: Replica: ecf68924cf1d44d4bfedec9f73009950, State: Running, Role: LEADER
I20260807 20:12:02.907033 22278 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:02.907013 22283 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf68924cf1d44d4bfedec9f73009950" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 40365 } }
I20260807 20:12:02.907362 22283 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecf68924cf1d44d4bfedec9f73009950. Latest consensus state: current_term: 1 leader_uuid: "ecf68924cf1d44d4bfedec9f73009950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf68924cf1d44d4bfedec9f73009950" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 40365 } } }
I20260807 20:12:02.907464 22283 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:02.907589 22285 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecf68924cf1d44d4bfedec9f73009950" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecf68924cf1d44d4bfedec9f73009950" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 40365 } } }
I20260807 20:12:02.907955 22289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:02.908552 22289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:02.907964 22285 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:02.909360 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:02.898249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.3.15.190:40365)
0807 20:12:02.898533 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:02.898542 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:02.898575 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:02.899203 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:02.899212 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:02.899275 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:02.899444 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:02.900041 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:02.901421 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:02.902252 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:02.902849 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:02.902961 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:02.905206 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:02.905209 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:02.905210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:02.905222 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:02.907162 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:02.907598 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:02.907601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:02.907602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:02.907700 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:02.907936 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:02.907937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:02.907938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:02.908278 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:02.908930 (+ 652us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:02.909017 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 20:12:02.909163 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":71,"threads_started":1}
I20260807 20:12:02.910917 22289 catalog_manager.cc:1347] Generated new cluster ID: 76f38bcec05d4fb0a71dd4a44acda2a4
I20260807 20:12:02.910962 22289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:02.926545 22289 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:02.927135 22289 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:02.931622 22289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950: Generated new TSK 0
I20260807 20:12:02.931831 22289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:12:02.966252 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:40365
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:03.052489 22305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:03.052692 22305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:03.052717 22305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:03.053921 22305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:03.053966 22305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:03.053992 22305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:12:03.055460 22305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:03.055724 22305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:03.055965 22305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:03.061282 22311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:03.061308 22305 server_base.cc:1034] running on GCE node
W20260807 20:12:03.061288 22310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:03.061290 22313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:03.061707 22305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:03.061987 22305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:03.063162 22305 hybrid_clock.cc:648] HybridClock initialized: now 1786133523063151 us; error 29 us; skew 500 ppm
Aug 07 20:12:03 dist-test-slave-1x4g krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133523, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:03.065984 22305 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:12:03.066413 22305 webserver.cc:460] Webserver started at http://127.3.15.129:37875/ using document root <none> and password file <none>
I20260807 20:12:03.066645 22305 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:03.066692 22305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:03.066793 22305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:03.067878 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "dc0a7d6e147642d39e7f0219cc72fc42"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "ceed05dd2a2079be068c9fd215ef9953"
server_key_iv: "a88a9fde408b2b5e7741222ce5a62c3b"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.068209 22305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "dc0a7d6e147642d39e7f0219cc72fc42"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "ceed05dd2a2079be068c9fd215ef9953"
server_key_iv: "a88a9fde408b2b5e7741222ce5a62c3b"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.069751 22305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:03.070609 22321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.070798 22305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:03.070922 22305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "dc0a7d6e147642d39e7f0219cc72fc42"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "ceed05dd2a2079be068c9fd215ef9953"
server_key_iv: "a88a9fde408b2b5e7741222ce5a62c3b"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.071007 22305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:03.085670 22305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:03.087745 22305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:03.087935 22305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:03.088213 22305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:03.088569 22305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:03.088629 22305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.088667 22305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:03.088717 22305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.095372 22305 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:33717
I20260807 20:12:03.095453 22442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:33717 every 8 connection(s)
I20260807 20:12:03.095836 22305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:03.102031 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22305
I20260807 20:12:03.102188 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:03.102412 3134 external_mini_cluster.cc:1428] Setting key e4c72ff7000a53942ca6b5f83fc5b379
Aug 07 20:12:03 dist-test-slave-1x4g krb5kdc[22191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133523, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:03.106331 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.096671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:44721 (local address 127.3.15.190:40365)
0807 20:12:03.096754 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.096757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.097023 (+ 266us) server_negotiation.cc:407] Connection header received
0807 20:12:03.097544 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.097545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.097556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.097621 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:03.098650 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.099237 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.100265 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.100490 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.100566 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.103068 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:03.103070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:03.103070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:03.103079 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:03.104691 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.105230 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.105232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.105232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.105302 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.105605 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.105606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.105606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.105738 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:03.105823 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.106133 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.106228 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 20:12:03.106545 22445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.096664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:40365 (local address 127.3.15.129:44721)
0807 20:12:03.096904 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.096943 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.097451 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.097695 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.097698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.098343 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:03.098557 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.098563 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.099319 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.099320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.100155 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.100158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.100579 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.100580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.100686 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.101329 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:03.101338 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:03.102952 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:03.104813 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:03.104821 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:03.104824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:03.105123 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:03.105399 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:03.105400 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:03.105400 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:03.105523 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:03.105813 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:03.105821 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:03.105984 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.106097 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.106277 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":147,"threads_started":1}
I20260807 20:12:03.107087 22443 heartbeater.cc:347] Connected to a master server at 127.3.15.190:40365
I20260807 20:12:03.107208 22443 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:03.107434 22443 heartbeater.cc:511] Master 127.3.15.190:40365 requested a full tablet report, sending...
I20260807 20:12:03.108089 22239 ts_manager.cc:194] Registered new tserver with Master: dc0a7d6e147642d39e7f0219cc72fc42 (127.3.15.129:33717)
I20260807 20:12:03.109139 22239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:44721
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:12:03.129801 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:40365
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:03.213855 22448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:03.214088 22448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:03.214114 22448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:03.215220 22448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:03.215261 22448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:03.215287 22448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:12:03.216711 22448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:03.216933 22448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:03.217140 22448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:03.222177 22457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:03.222179 22455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:03.222220 22454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:03.222453 22448 server_base.cc:1034] running on GCE node
I20260807 20:12:03.222749 22448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:03.222988 22448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:03.224153 22448 hybrid_clock.cc:648] HybridClock initialized: now 1786133523224131 us; error 42 us; skew 500 ppm
Aug 07 20:12:03 dist-test-slave-1x4g krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133523, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:03.226797 22448 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:12:03.227203 22448 webserver.cc:460] Webserver started at http://127.3.15.130:42103/ using document root <none> and password file <none>
I20260807 20:12:03.227396 22448 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:03.227437 22448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:03.227530 22448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:03.228581 22448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "45d02141adb74d4b95a3dce8957b2aae"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "6a3ffdd9f9890a4049671954fc7795a6"
server_key_iv: "a9496710b1eac0120aa4e68a89cb3281"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.228971 22448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "45d02141adb74d4b95a3dce8957b2aae"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "6a3ffdd9f9890a4049671954fc7795a6"
server_key_iv: "a9496710b1eac0120aa4e68a89cb3281"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.230314 22448 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:03.231027 22464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.231218 22448 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:03.231313 22448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "45d02141adb74d4b95a3dce8957b2aae"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "6a3ffdd9f9890a4049671954fc7795a6"
server_key_iv: "a9496710b1eac0120aa4e68a89cb3281"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.231370 22448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:03.245667 22448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:03.247713 22448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:03.247833 22448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:03.248027 22448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:03.248368 22448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:03.248412 22448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.248467 22448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:03.248502 22448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.254345 22448 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:40749
I20260807 20:12:03.254427 22587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:40749 every 8 connection(s)
I20260807 20:12:03.254779 22448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:03.255254 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22448
I20260807 20:12:03.255378 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:03.255643 3134 external_mini_cluster.cc:1428] Setting key 4015d7f3d3a3206a634d337ed65dbf8c
Aug 07 20:12:03 dist-test-slave-1x4g krb5kdc[22191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133523, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
I20260807 20:12:03.266058 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.255564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:33945 (local address 127.3.15.190:40365)
0807 20:12:03.255634 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.255637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.255884 (+ 247us) server_negotiation.cc:407] Connection header received
0807 20:12:03.256462 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.256463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.256475 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.256547 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:03.257625 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.258433 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.259933 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.260283 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.260359 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.262849 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:03.262850 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:03.262851 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:03.262860 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:03.264539 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.264989 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.264991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.264991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.265060 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.265327 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.265328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.265329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.265487 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:03.265568 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.265832 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.265928 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 20:12:03.266112 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.255549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:40365 (local address 127.3.15.130:33945)
0807 20:12:03.255736 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.255778 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.256352 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.256625 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.256628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.257312 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:03.257539 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.257544 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.258538 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.258540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.259846 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.259849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.260368 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.260369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.260469 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.261084 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:03.261093 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:03.262747 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:03.264629 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:03.264636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:03.264638 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:03.264914 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:03.265146 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:03.265147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:03.265147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:03.265247 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:03.265555 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:03.265573 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:03.265689 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.265766 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.265888 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":115,"threads_started":1}
I20260807 20:12:03.266619 22588 heartbeater.cc:347] Connected to a master server at 127.3.15.190:40365
I20260807 20:12:03.266713 22588 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:03.266894 22588 heartbeater.cc:511] Master 127.3.15.190:40365 requested a full tablet report, sending...
I20260807 20:12:03.267316 22239 ts_manager.cc:194] Registered new tserver with Master: 45d02141adb74d4b95a3dce8957b2aae (127.3.15.130:40749)
I20260807 20:12:03.267696 22239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:33945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:12:03.283998 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:40365
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:34451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:03.369230 22593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:03.369467 22593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:03.369493 22593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:03.370713 22593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:03.370769 22593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:03.370833 22593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:12:03.372390 22593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:34451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:40365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:03.372673 22593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:03.372964 22593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:03.378309 22599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:03.378322 22598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:03.378340 22593 server_base.cc:1034] running on GCE node
W20260807 20:12:03.378527 22601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:03.378753 22593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:03.379052 22593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:03.380206 22593 hybrid_clock.cc:648] HybridClock initialized: now 1786133523380199 us; error 30 us; skew 500 ppm
Aug 07 20:12:03 dist-test-slave-1x4g krb5kdc[22191](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133523, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:03.382786 22593 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:12:03.383229 22593 webserver.cc:460] Webserver started at http://127.3.15.131:34513/ using document root <none> and password file <none>
I20260807 20:12:03.383474 22593 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:03.383544 22593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:03.383718 22593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:03.385003 22593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "3a998e86ccee4b00af6c60b59e11f90c"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "86badcd86850242b42c81b8e0d203970"
server_key_iv: "2ccf6d23fd689371c25d43a5a59f619c"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.385407 22593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "3a998e86ccee4b00af6c60b59e11f90c"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "86badcd86850242b42c81b8e0d203970"
server_key_iv: "2ccf6d23fd689371c25d43a5a59f619c"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.386813 22593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:03.387549 22609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.387733 22593 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:03.387835 22593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "3a998e86ccee4b00af6c60b59e11f90c"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "86badcd86850242b42c81b8e0d203970"
server_key_iv: "2ccf6d23fd689371c25d43a5a59f619c"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.387923 22593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:03.406414 22593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:03.409209 22593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:03.409330 22593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:03.409586 22593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:03.409927 22593 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:03.409986 22593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.410030 22593 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:03.410058 22593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.415825 22593 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:38819
I20260807 20:12:03.415922 22733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:38819 every 8 connection(s)
I20260807 20:12:03.416272 22593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:03.419682 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22593
I20260807 20:12:03.419776 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:03.419998 3134 external_mini_cluster.cc:1428] Setting key ac90f6f2427a0e0168e231a4270a135a
Aug 07 20:12:03 dist-test-slave-1x4g krb5kdc[22191](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133523, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:03.426337 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.417105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:36095 (local address 127.3.15.190:40365)
0807 20:12:03.417170 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.417172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.417399 (+ 227us) server_negotiation.cc:407] Connection header received
0807 20:12:03.417963 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.417965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.417978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.418036 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:03.419031 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.419570 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.421174 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.421378 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.421465 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.423648 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:03.423650 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:03.423651 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:03.423658 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:03.424922 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.425338 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.425340 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.425340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.425405 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.425669 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.425670 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.425670 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.425793 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:03.425871 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.426160 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.426246 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:12:03.426518 22737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.417093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:40365 (local address 127.3.15.131:36095)
0807 20:12:03.417273 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.417313 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.417842 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.418114 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.418119 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.418720 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:03.418928 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.418933 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.419677 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.419678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.421100 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.421103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.421476 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.421477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.421628 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.422167 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:03.422180 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:03.423536 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:03.425015 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:03.425023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:03.425025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:03.425257 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:03.425474 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:03.425475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:03.425475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:03.425590 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:03.425862 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:03.425870 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:03.426013 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.426116 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.426278 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":88,"threads_started":1}
I20260807 20:12:03.427073 22735 heartbeater.cc:347] Connected to a master server at 127.3.15.190:40365
I20260807 20:12:03.427192 22735 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:03.427382 22735 heartbeater.cc:511] Master 127.3.15.190:40365 requested a full tablet report, sending...
I20260807 20:12:03.427776 22239 ts_manager.cc:194] Registered new tserver with Master: 3a998e86ccee4b00af6c60b59e11f90c (127.3.15.131:38819)
I20260807 20:12:03.428160 22239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:36095
I20260807 20:12:03.432080 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:03.440208 22281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.433504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.3.15.190:40365)
0807 20:12:03.433581 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.433584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.433703 (+ 119us) server_negotiation.cc:407] Connection header received
0807 20:12:03.433779 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.433780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.433790 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.433835 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:03.434515 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.434987 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.435803 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.436019 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.436107 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.437634 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:03.437636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:03.437636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:03.437644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:03.438901 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.439317 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.439318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.439318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.439380 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:03.439612 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:03.439613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:03.439614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:03.439758 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:03.439881 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.440032 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.440109 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 20:12:03.442036 22239 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:03.443809 22239 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:03.450388 22746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.446272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:38819 (local address 127.0.0.1:59928)
0807 20:12:03.446399 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.446447 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.446517 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.446861 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.446864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.446867 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:03.446944 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.446946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.448086 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.448087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.449154 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.449156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.450064 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.450065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.450147 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.450169 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.450258 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.450313 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
I20260807 20:12:03.450400 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.446393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.3.15.131:38819)
0807 20:12:03.446613 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.446618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.446628 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:03.446667 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.446669 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.446685 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.446758 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:03.447137 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.447979 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.449252 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.449971 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.450053 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.450128 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.450278 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.450320 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":104,"threads_started":1}
I20260807 20:12:03.450966 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.446670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:40749 (local address 127.0.0.1:34954)
0807 20:12:03.447049 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.447079 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.447127 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.447398 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.447399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.447401 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:03.447441 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.447443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.448706 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.448707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.449696 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.449700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.450734 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.450735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.450799 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.450804 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.450854 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.450896 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":68,"threads_started":1}
I20260807 20:12:03.451012 22751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.446802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.3.15.130:40749)
0807 20:12:03.447002 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.447007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.447185 (+ 178us) server_negotiation.cc:407] Connection header received
0807 20:12:03.447233 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.447235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.447251 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.447319 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:03.447527 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.448603 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.449801 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.450573 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.450639 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.450697 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.450890 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.450938 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260807 20:12:03.451263 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.446639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42374 (local address 127.3.15.129:33717)
0807 20:12:03.446841 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.446845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.446975 (+ 130us) server_negotiation.cc:407] Connection header received
0807 20:12:03.447097 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.447099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.447116 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.447192 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:03.447397 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.448545 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.449874 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.450766 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.450829 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.450883 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.451141 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.451177 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":82,"threads_started":1}
I20260807 20:12:03.451195 22662 tablet_service.cc:1467] Processing CreateTablet for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f (DEFAULT_TABLE table=test-table [id=4220ae18dc01406e865185a13ef29d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:03.451525 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.446487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:33717 (local address 127.0.0.1:42374)
0807 20:12:03.446882 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.446890 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.447005 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.447283 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.447284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.447285 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:03.447332 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.447334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.448670 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.448671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.449783 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.449786 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.450915 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.450916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.450972 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.450976 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.451025 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.451451 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":2,"thread_start_us":86,"threads_started":1}
I20260807 20:12:03.451582 22662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:03.451795 22515 tablet_service.cc:1467] Processing CreateTablet for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f (DEFAULT_TABLE table=test-table [id=4220ae18dc01406e865185a13ef29d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:03.452126 22515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:03.453169 22372 tablet_service.cc:1467] Processing CreateTablet for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f (DEFAULT_TABLE table=test-table [id=4220ae18dc01406e865185a13ef29d6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:03.453477 22372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:03.455294 22752 tablet_bootstrap.cc:492] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Bootstrap starting.
I20260807 20:12:03.456081 22752 tablet_bootstrap.cc:654] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:03.456410 22753 tablet_bootstrap.cc:492] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: Bootstrap starting.
I20260807 20:12:03.456904 22752 log.cc:824] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:03.457007 22754 tablet_bootstrap.cc:492] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: Bootstrap starting.
I20260807 20:12:03.457144 22753 tablet_bootstrap.cc:654] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:03.457711 22753 log.cc:824] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:03.457742 22754 tablet_bootstrap.cc:654] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:03.457959 22752 tablet_bootstrap.cc:492] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: No bootstrap required, opened a new log
I20260807 20:12:03.458035 22752 ts_tablet_manager.cc:1397] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 20:12:03.458321 22754 log.cc:824] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:03.458505 22753 tablet_bootstrap.cc:492] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: No bootstrap required, opened a new log
I20260807 20:12:03.458570 22753 ts_tablet_manager.cc:1397] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:03.459169 22754 tablet_bootstrap.cc:492] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: No bootstrap required, opened a new log
I20260807 20:12:03.459242 22754 ts_tablet_manager.cc:1397] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:03.459810 22752 raft_consensus.cc:348] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.459980 22752 raft_consensus.cc:374] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:03.460026 22752 raft_consensus.cc:729] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a998e86ccee4b00af6c60b59e11f90c, State: Initialized, Role: FOLLOWER
I20260807 20:12:03.460156 22752 consensus_queue.cc:260] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.460451 22752 ts_tablet_manager.cc:1428] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260807 20:12:03.460599 22735 heartbeater.cc:503] Master 127.3.15.190:40365 was elected leader, sending a full tablet report...
I20260807 20:12:03.460530 22753 raft_consensus.cc:348] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.460708 22753 raft_consensus.cc:374] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:03.460753 22753 raft_consensus.cc:729] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d02141adb74d4b95a3dce8957b2aae, State: Initialized, Role: FOLLOWER
I20260807 20:12:03.460892 22753 consensus_queue.cc:260] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.461176 22753 ts_tablet_manager.cc:1428] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:03.461197 22588 heartbeater.cc:503] Master 127.3.15.190:40365 was elected leader, sending a full tablet report...
I20260807 20:12:03.461288 22754 raft_consensus.cc:348] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.461467 22754 raft_consensus.cc:374] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:03.461505 22754 raft_consensus.cc:729] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc0a7d6e147642d39e7f0219cc72fc42, State: Initialized, Role: FOLLOWER
I20260807 20:12:03.461621 22754 consensus_queue.cc:260] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.461941 22443 heartbeater.cc:503] Master 127.3.15.190:40365 was elected leader, sending a full tablet report...
I20260807 20:12:03.461933 22754 ts_tablet_manager.cc:1428] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260807 20:12:03.505364 22589 tablet.cc:2369] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:03.596633 22444 tablet.cc:2369] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:03.666873 22736 tablet.cc:2369] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:03.670374 22761 raft_consensus.cc:482] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:03.670516 22761 raft_consensus.cc:504] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.670919 22761 leader_election.cc:288] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a998e86ccee4b00af6c60b59e11f90c (127.3.15.131:38819), 45d02141adb74d4b95a3dce8957b2aae (127.3.15.130:40749)
I20260807 20:12:03.674317 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.671241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:34371 (local address 127.3.15.131:38819)
0807 20:12:03.671313 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.671315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.671345 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:12:03.671426 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.671428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.671438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.671520 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:03.671700 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.672187 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.673503 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.673890 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.673980 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.674006 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.674203 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.674253 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 20:12:03.674352 22751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.671441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:36937 (local address 127.3.15.130:40749)
0807 20:12:03.671501 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.671503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.671513 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:03.671545 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.671546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.671557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.671622 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:03.671794 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.672265 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.673544 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.673931 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.674044 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.674053 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.674246 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.674286 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 20:12:03.674325 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.671129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:38819 (local address 127.3.15.129:34371)
0807 20:12:03.671259 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.671271 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.671332 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.671533 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.671534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.671537 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:03.671623 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.671625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.672282 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.672283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.673417 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.673420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.673991 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.673992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.674080 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.674085 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.674148 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.674201 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260807 20:12:03.674328 22763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.671340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:40749 (local address 127.3.15.129:36937)
0807 20:12:03.671440 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.671449 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:03.671488 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:03.671627 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:03.671628 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:03.671630 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:03.671717 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.671719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.672351 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.672352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.673438 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:03.673441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.674052 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.674053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:03.674114 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.674119 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:03.674193 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:12:03.674219 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":63,"threads_started":1}
I20260807 20:12:03.674738 22684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a26ee00b3e4c2aacea99bd6e6fcd4f" candidate_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a998e86ccee4b00af6c60b59e11f90c" is_pre_election: true
I20260807 20:12:03.674914 22684 raft_consensus.cc:2454] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0a7d6e147642d39e7f0219cc72fc42 in term 0.
I20260807 20:12:03.675212 22323 leader_election.cc:302] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a998e86ccee4b00af6c60b59e11f90c, dc0a7d6e147642d39e7f0219cc72fc42; no voters:
I20260807 20:12:03.675335 22761 raft_consensus.cc:2781] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:03.675395 22761 raft_consensus.cc:482] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:03.675421 22761 raft_consensus.cc:3037] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:03.675359 22537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a26ee00b3e4c2aacea99bd6e6fcd4f" candidate_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d02141adb74d4b95a3dce8957b2aae" is_pre_election: true
I20260807 20:12:03.675540 22537 raft_consensus.cc:2454] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc0a7d6e147642d39e7f0219cc72fc42 in term 0.
I20260807 20:12:03.676224 22761 raft_consensus.cc:504] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.676362 22761 leader_election.cc:288] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [CANDIDATE]: Term 1 election: Requested vote from peers 3a998e86ccee4b00af6c60b59e11f90c (127.3.15.131:38819), 45d02141adb74d4b95a3dce8957b2aae (127.3.15.130:40749)
I20260807 20:12:03.676532 22684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a26ee00b3e4c2aacea99bd6e6fcd4f" candidate_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a998e86ccee4b00af6c60b59e11f90c"
I20260807 20:12:03.676627 22684 raft_consensus.cc:3037] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:03.676853 22537 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3a26ee00b3e4c2aacea99bd6e6fcd4f" candidate_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d02141adb74d4b95a3dce8957b2aae"
I20260807 20:12:03.676941 22537 raft_consensus.cc:3037] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:03.677461 22684 raft_consensus.cc:2454] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0a7d6e147642d39e7f0219cc72fc42 in term 1.
I20260807 20:12:03.677647 22323 leader_election.cc:302] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a998e86ccee4b00af6c60b59e11f90c, dc0a7d6e147642d39e7f0219cc72fc42; no voters:
I20260807 20:12:03.677757 22761 raft_consensus.cc:2781] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:03.677816 22537 raft_consensus.cc:2454] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc0a7d6e147642d39e7f0219cc72fc42 in term 1.
I20260807 20:12:03.677943 22761 raft_consensus.cc:686] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [term 1 LEADER]: Becoming Leader. State: Replica: dc0a7d6e147642d39e7f0219cc72fc42, State: Running, Role: LEADER
I20260807 20:12:03.678058 22761 consensus_queue.cc:237] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } }
I20260807 20:12:03.678903 22239 catalog_manager.cc:5495] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc0a7d6e147642d39e7f0219cc72fc42 (127.3.15.129). New cstate: current_term: 1 leader_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc0a7d6e147642d39e7f0219cc72fc42" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 33717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:03.734340 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.731646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.3.15.129:33717)
0807 20:12:03.731727 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.731728 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.731738 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:03.731783 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.731783 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.731808 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.731942 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:03.732124 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.732673 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.733516 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.733752 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.733808 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.733957 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:03.734070 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:03.734163 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.734266 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.734279 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:12:03.736292 22537 raft_consensus.cc:1261] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 1 FOLLOWER]: Refusing update from remote peer dc0a7d6e147642d39e7f0219cc72fc42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:03.736286 22684 raft_consensus.cc:1261] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 1 FOLLOWER]: Refusing update from remote peer dc0a7d6e147642d39e7f0219cc72fc42: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:03.736690 22761 consensus_queue.cc:1035] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a998e86ccee4b00af6c60b59e11f90c" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 38819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:03.736892 22764 consensus_queue.cc:1035] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P dc0a7d6e147642d39e7f0219cc72fc42 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d02141adb74d4b95a3dce8957b2aae" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:03.739157 22767 mvcc.cc:205] Tried to move back new op lower bound from 7316002913221083136 to 7316002912986017792. Current Snapshot: MvccSnapshot[applied={T|T < 7316002913221083136}]
I20260807 20:12:03.741811 22238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:03.741904 22238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:03.742741 22238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ecf68924cf1d44d4bfedec9f73009950: Sending DeleteTablet for 3 replicas of tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f
I20260807 20:12:03.743185 22372 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:03 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:42374
I20260807 20:12:03.743252 22662 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:03 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:59928
I20260807 20:12:03.743418 22515 tablet_service.cc:1514] Processing DeleteTablet for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:03 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:34954
I20260807 20:12:03.743485 22777 tablet_replica.cc:330] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: stopping tablet replica
I20260807 20:12:03.743587 22777 raft_consensus.cc:2229] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:03.743708 22777 raft_consensus.cc:2258] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:03.743693 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22305
I20260807 20:12:03.746071 22777 ts_tablet_manager.cc:1905] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:03.748006 22777 ts_tablet_manager.cc:1918] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:03.748073 22777 log.cc:1196] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/e3a26ee00b3e4c2aacea99bd6e6fcd4f
I20260807 20:12:03.748366 22777 ts_tablet_manager.cc:1939] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 3a998e86ccee4b00af6c60b59e11f90c: Deleting consensus metadata
I20260807 20:12:03.748809 22223 catalog_manager.cc:4841] TS 3a998e86ccee4b00af6c60b59e11f90c (127.3.15.131:38819): tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f (table test-table [id=4220ae18dc01406e865185a13ef29d6d]) successfully deleted
W20260807 20:12:03.749989 22223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:33717 (error 108)
I20260807 20:12:03.750550 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.750423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:33717 (local address 127.0.0.1:42398)
0807 20:12:03.750477 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.750522 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:33717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 20:12:03.750576 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22448
I20260807 20:12:03.750592 22779 tablet_replica.cc:330] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae: stopping tablet replica
I20260807 20:12:03.750700 22779 raft_consensus.cc:2229] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 1 FOLLOWER]: Raft consensus shutting down.
W20260807 20:12:03.750787 22223 catalog_manager.cc:4584] TS dc0a7d6e147642d39e7f0219cc72fc42 (127.3.15.129:33717): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e3a26ee00b3e4c2aacea99bd6e6fcd4f: Network error: Client connection negotiation failed: client connection to 127.3.15.129:33717: connect: Connection refused (error 111)
I20260807 20:12:03.750819 22779 raft_consensus.cc:2258] T e3a26ee00b3e4c2aacea99bd6e6fcd4f P 45d02141adb74d4b95a3dce8957b2aae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:03.757041 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22593
I20260807 20:12:03.757074 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.756914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:40749 (local address 127.0.0.1:34970)
0807 20:12:03.756988 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:03.757021 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.130:40749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 20:12:03.762681 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22204
2026-08-07T20:12:03Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2185 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T20:12:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:03Z Disabled control of system clock
I20260807 20:12:03.792191 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44973
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:44973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:03.874830 22784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:03.875068 22784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:03.875090 22784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:03.876257 22784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:03.876297 22784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:03.876309 22784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:03.876322 22784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:03.876334 22784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:03.877707 22784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:44973
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:44973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:03.877913 22784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:03.878170 22784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:03.883193 22789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:03.883236 22792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:03.883271 22790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:03.883276 22784 server_base.cc:1034] running on GCE node
I20260807 20:12:03.883723 22784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:03.884068 22784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:03.885231 22784 hybrid_clock.cc:648] HybridClock initialized: now 1786133523885216 us; error 44 us; skew 500 ppm
I20260807 20:12:03.891213 22784 webserver.cc:460] Webserver started at http://127.3.15.190:32981/ using document root <none> and password file <none>
I20260807 20:12:03.891507 22784 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:03.891582 22784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:03.891748 22784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:03.892932 22784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "80aebf2a866c46e49f32383bd6b8a6ad"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "d825c45c3dd5005a45be02e946338dba"
server_key_iv: "a0efb4f96b2722e64894bb10e5dec5a1"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.893318 22784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "80aebf2a866c46e49f32383bd6b8a6ad"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "d825c45c3dd5005a45be02e946338dba"
server_key_iv: "a0efb4f96b2722e64894bb10e5dec5a1"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.894655 22784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:03.895404 22798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:03.895574 22784 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:03.895684 22784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "80aebf2a866c46e49f32383bd6b8a6ad"
format_stamp: "Formatted at 2026-08-07 20:12:03 on dist-test-slave-1x4g"
server_key: "d825c45c3dd5005a45be02e946338dba"
server_key_iv: "a0efb4f96b2722e64894bb10e5dec5a1"
server_key_version: "encryptionkey@0"
I20260807 20:12:03.895758 22784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:03.933068 22784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:03.933454 22784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:03.933626 22784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:03.938401 22784 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:44973
I20260807 20:12:03.938438 22853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:44973 every 8 connection(s)
I20260807 20:12:03.938831 22784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:03.939896 22854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:03.942389 22854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad: Bootstrap starting.
I20260807 20:12:03.942966 22854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:03.943226 22854 log.cc:824] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:03.944021 22854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad: No bootstrap required, opened a new log
I20260807 20:12:03.945772 22854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44973 } }
I20260807 20:12:03.945920 22854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:03.945942 22854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80aebf2a866c46e49f32383bd6b8a6ad, State: Initialized, Role: FOLLOWER
I20260807 20:12:03.946051 22854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44973 } }
I20260807 20:12:03.946131 22854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:03.946170 22854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:03.946244 22854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:03.946800 22854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44973 } }
I20260807 20:12:03.946921 22854 leader_election.cc:302] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80aebf2a866c46e49f32383bd6b8a6ad; no voters:
I20260807 20:12:03.947124 22854 leader_election.cc:288] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:03.947266 22857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:03.947301 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22784
I20260807 20:12:03.947412 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:03.947422 22854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:03.947490 22857 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [term 1 LEADER]: Becoming Leader. State: Replica: 80aebf2a866c46e49f32383bd6b8a6ad, State: Running, Role: LEADER
I20260807 20:12:03.947588 22857 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44973 } }
I20260807 20:12:03.947661 3134 external_mini_cluster.cc:1428] Setting key f20fee7617ff2a706f9428c36c19a790
I20260807 20:12:03.948163 22858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44973 } } }
I20260807 20:12:03.948263 22858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:03.948511 22859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80aebf2a866c46e49f32383bd6b8a6ad. Latest consensus state: current_term: 1 leader_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80aebf2a866c46e49f32383bd6b8a6ad" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 44973 } } }
I20260807 20:12:03.948635 22859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:03.948992 22869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:03.949527 22869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:03.951195 22869 catalog_manager.cc:1347] Generated new cluster ID: 7721959287b84af7ac9ce3daf97a4d3e
I20260807 20:12:03.951251 22869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:03.953490 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:03.948185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.3.15.190:44973)
0807 20:12:03.948521 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:03.948531 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:03.948589 (+ 58us) server_negotiation.cc:407] Connection header received
0807 20:12:03.948718 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:03.948727 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:03.948830 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:03.948986 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:03.949269 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.950389 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.951413 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:03.951849 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:03.951977 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:03.952740 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:03.952744 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:03.952745 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:03.952971 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:03.953006 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:03.953017 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:03.953102 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:03.953174 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 20:12:03.953261 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":252,"thread_start_us":87,"threads_started":1}
I20260807 20:12:03.968492 22869 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:03.969126 22869 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:03.973026 22869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad: Generated new TSK 0
I20260807 20:12:03.973218 22869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:03.980629 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44973
--builtin_ntp_servers=127.3.15.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:04.083119 22881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:04.083346 22881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:04.083384 22881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:04.085107 22881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:04.085165 22881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:04.085208 22881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:12:04.086544 22881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:04.086822 22881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:04.087127 22881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:04.093092 22887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:04.093106 22889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:04.093386 22881 server_base.cc:1034] running on GCE node
W20260807 20:12:04.093228 22886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:04.093683 22881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:04.093922 22881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:04.095085 22881 hybrid_clock.cc:648] HybridClock initialized: now 1786133524095073 us; error 30 us; skew 500 ppm
I20260807 20:12:04.096483 22881 webserver.cc:460] Webserver started at http://127.3.15.129:35453/ using document root <none> and password file <none>
I20260807 20:12:04.096807 22881 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:04.096886 22881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:04.097034 22881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:04.098110 22881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "da2913ef9f994a8faae10475ec5f9f69"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "cc0d7a86b442afa19e657df7e0b024d6"
server_key_iv: "76c0f06a31293b58891dc1f07222f101"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.098500 22881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "da2913ef9f994a8faae10475ec5f9f69"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "cc0d7a86b442afa19e657df7e0b024d6"
server_key_iv: "76c0f06a31293b58891dc1f07222f101"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.099927 22881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:04.100687 22896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.100955 22881 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:04.101052 22881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "da2913ef9f994a8faae10475ec5f9f69"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "cc0d7a86b442afa19e657df7e0b024d6"
server_key_iv: "76c0f06a31293b58891dc1f07222f101"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.101130 22881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:04.124152 22881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:04.124449 22881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:04.124555 22881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:04.124753 22881 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:04.125185 22881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:04.125221 22881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.125279 22881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:04.125304 22881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.131247 22881 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:41023
I20260807 20:12:04.131398 23016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:41023 every 8 connection(s)
I20260807 20:12:04.131695 22881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:04.136981 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.132546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:55537 (local address 127.3.15.190:44973)
0807 20:12:04.132643 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.132646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.132877 (+ 231us) server_negotiation.cc:407] Connection header received
0807 20:12:04.133389 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.133390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.133404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.133453 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:04.134117 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.134621 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.135611 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.135800 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.135850 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.136648 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:04.136649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:04.136649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:04.136654 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:04.136669 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:04.136673 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:04.136716 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.136877 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.136918 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 20:12:04.137171 23019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.132533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44973 (local address 127.3.15.129:55537)
0807 20:12:04.132744 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.132799 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.133301 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.133516 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.133519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.133528 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:04.134040 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.134044 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.134702 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.134703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.135521 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.135524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.135875 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.135876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.135994 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.136493 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:04.136493 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:04.136523 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:04.136547 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:04.136730 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:04.136734 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:04.136737 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.136871 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.136958 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
I20260807 20:12:04.137403 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 22881
I20260807 20:12:04.137501 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:04.137727 3134 external_mini_cluster.cc:1428] Setting key e62750ac9e68858bb44f57ddca9a0efc
I20260807 20:12:04.137876 23017 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44973
I20260807 20:12:04.137986 23017 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:04.138190 23017 heartbeater.cc:511] Master 127.3.15.190:44973 requested a full tablet report, sending...
I20260807 20:12:04.138756 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44973
--builtin_ntp_servers=127.3.15.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:12:04.138783 22815 ts_manager.cc:194] Registered new tserver with Master: da2913ef9f994a8faae10475ec5f9f69 (127.3.15.129:41023)
I20260807 20:12:04.139698 22815 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.129:55537
W20260807 20:12:04.223892 23020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:04.224193 23020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:04.224220 23020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:04.225364 23020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:04.225399 23020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:04.225423 23020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:12:04.226697 23020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:04.226886 23020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:04.227138 23020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:04.232479 23028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:04.232600 23026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:04.232620 23025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:04.232885 23020 server_base.cc:1034] running on GCE node
I20260807 20:12:04.233179 23020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:04.233425 23020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:04.234699 23020 hybrid_clock.cc:648] HybridClock initialized: now 1786133524234684 us; error 31 us; skew 500 ppm
I20260807 20:12:04.236202 23020 webserver.cc:460] Webserver started at http://127.3.15.130:41285/ using document root <none> and password file <none>
I20260807 20:12:04.236469 23020 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:04.236541 23020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:04.236709 23020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:04.237833 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "801a2564aac446df8d937e927c8bd565"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "120b31156e32f3b3d3f4387cd52c3faa"
server_key_iv: "9f4bace3146c81cd5cfe9d3775c70294"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.238217 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "801a2564aac446df8d937e927c8bd565"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "120b31156e32f3b3d3f4387cd52c3faa"
server_key_iv: "9f4bace3146c81cd5cfe9d3775c70294"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.239537 23020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:04.240250 23034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.240428 23020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:04.240536 23020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "801a2564aac446df8d937e927c8bd565"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "120b31156e32f3b3d3f4387cd52c3faa"
server_key_iv: "9f4bace3146c81cd5cfe9d3775c70294"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.240614 23020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:04.259650 23020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:04.260008 23020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:04.260174 23020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:04.260443 23020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:04.260864 23020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:04.260919 23020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.260964 23020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:04.260993 23020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.267019 23020 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:42475
I20260807 20:12:04.267063 23156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:42475 every 8 connection(s)
I20260807 20:12:04.267429 23020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:04.272624 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.268225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:46307 (local address 127.3.15.190:44973)
0807 20:12:04.268298 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.268300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.268528 (+ 228us) server_negotiation.cc:407] Connection header received
0807 20:12:04.269109 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.269111 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.269121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.269184 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:04.269845 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.270344 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.271306 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.271486 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.271538 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.272398 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:04.272399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:04.272399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:04.272404 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:04.272417 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:04.272421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:04.272462 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.272538 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.272562 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:12:04.272845 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.268209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44973 (local address 127.3.15.130:46307)
0807 20:12:04.268391 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.268434 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.269015 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.269241 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.269244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.269253 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:04.269767 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.269771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.270429 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.270429 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.271224 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.271226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.271561 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.271562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.271671 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.272256 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:04.272257 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:04.272292 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:04.272318 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:04.272472 (+ 154us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:04.272474 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:04.272477 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.272526 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.272609 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
I20260807 20:12:04.273398 23157 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44973
I20260807 20:12:04.273504 23157 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:04.273695 23157 heartbeater.cc:511] Master 127.3.15.190:44973 requested a full tablet report, sending...
I20260807 20:12:04.274075 22815 ts_manager.cc:194] Registered new tserver with Master: 801a2564aac446df8d937e927c8bd565 (127.3.15.130:42475)
I20260807 20:12:04.274461 22815 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.130:46307
I20260807 20:12:04.274909 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23020
I20260807 20:12:04.275013 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:04.275228 3134 external_mini_cluster.cc:1428] Setting key 38211b3f4418d999f9de1256ff061580
I20260807 20:12:04.276254 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44973
--builtin_ntp_servers=127.3.15.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:04.360445 23160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:04.360711 23160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:04.360761 23160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:04.362000 23160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:04.362079 23160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:04.362125 23160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:12:04.363500 23160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:44973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:04.363776 23160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:04.364112 23160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:04.369293 23170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:04.369298 23168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:04.369371 23166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:04.369407 23160 server_base.cc:1034] running on GCE node
I20260807 20:12:04.369763 23160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:04.370015 23160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:04.371207 23160 hybrid_clock.cc:648] HybridClock initialized: now 1786133524371190 us; error 29 us; skew 500 ppm
I20260807 20:12:04.372553 23160 webserver.cc:460] Webserver started at http://127.3.15.131:40569/ using document root <none> and password file <none>
I20260807 20:12:04.372887 23160 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:04.372959 23160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:04.373116 23160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:04.374195 23160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "7e720622c3234a29806bb1dcf352b77e"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "bc5aa4c6e7f90da4465659e5447f8374"
server_key_iv: "7790644b4f6ff4dede34a97d7ce62251"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.374588 23160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "7e720622c3234a29806bb1dcf352b77e"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "bc5aa4c6e7f90da4465659e5447f8374"
server_key_iv: "7790644b4f6ff4dede34a97d7ce62251"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.375901 23160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:04.376605 23176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.376829 23160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:04.376938 23160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "7e720622c3234a29806bb1dcf352b77e"
format_stamp: "Formatted at 2026-08-07 20:12:04 on dist-test-slave-1x4g"
server_key: "bc5aa4c6e7f90da4465659e5447f8374"
server_key_iv: "7790644b4f6ff4dede34a97d7ce62251"
server_key_version: "encryptionkey@0"
I20260807 20:12:04.377025 23160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:04.398742 23160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:04.399106 23160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:04.399276 23160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:04.399569 23160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:04.399979 23160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:04.400051 23160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.400099 23160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:04.400128 23160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:04.407086 23160 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:36991
I20260807 20:12:04.407128 23300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:36991 every 8 connection(s)
I20260807 20:12:04.407517 23160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:04.411954 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23160
I20260807 20:12:04.412083 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:04.412276 3134 external_mini_cluster.cc:1428] Setting key 96708eeccdd3278e6c7c73cf6e55a95e
I20260807 20:12:04.413036 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.408368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:38335 (local address 127.3.15.190:44973)
0807 20:12:04.408432 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.408434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.408693 (+ 259us) server_negotiation.cc:407] Connection header received
0807 20:12:04.409240 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.409242 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.409256 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.409306 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:04.409996 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.410482 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.411424 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.411612 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.411662 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.412619 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:04.412620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:04.412621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:04.412626 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:04.412666 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:04.412671 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:04.412722 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.412903 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.412941 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:12:04.413283 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.408356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:44973 (local address 127.3.15.131:38335)
0807 20:12:04.408565 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.408601 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.409146 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.409379 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.409382 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.409391 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:04.409919 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.409924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.410568 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.410568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.411348 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.411350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.411683 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.411684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.411863 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.412437 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:04.412437 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:04.412484 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:04.412508 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:04.412743 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:04.412751 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:04.412754 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.412886 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.413017 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":101,"threads_started":1}
I20260807 20:12:04.413765 23301 heartbeater.cc:347] Connected to a master server at 127.3.15.190:44973
I20260807 20:12:04.413872 23301 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:04.414069 23301 heartbeater.cc:511] Master 127.3.15.190:44973 requested a full tablet report, sending...
I20260807 20:12:04.414449 22815 ts_manager.cc:194] Registered new tserver with Master: 7e720622c3234a29806bb1dcf352b77e (127.3.15.131:36991)
I20260807 20:12:04.414784 22815 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.131:38335
I20260807 20:12:04.424880 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:04.429117 22867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.426297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.3.15.190:44973)
0807 20:12:04.426382 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.426385 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.426419 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:12:04.426486 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.426487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.426496 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.426534 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:04.426684 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.427194 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.427889 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.428065 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.428113 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.428767 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:04.428769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:04.428769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:04.428815 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:04.428833 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:04.428838 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:04.428930 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.429003 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.429014 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 20:12:04.430608 22815 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:04.432184 22815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:04.438823 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.434493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55214 (local address 127.3.15.131:36991)
0807 20:12:04.434692 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.434698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.434897 (+ 199us) server_negotiation.cc:407] Connection header received
0807 20:12:04.434944 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.434947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.434964 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.435035 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:04.435406 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.436165 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.437751 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.438256 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.438352 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.438463 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.438692 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.438737 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260807 20:12:04.438810 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.434399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:36991 (local address 127.0.0.1:55214)
0807 20:12:04.434738 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.434749 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.434812 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.435211 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.435212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.435214 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:04.435252 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.435255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.436256 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.436257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.437643 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.437646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.438490 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.438491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.438579 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.438590 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.438674 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.438732 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":22,"thread_start_us":69,"threads_started":1}
I20260807 20:12:04.438990 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.434851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38768 (local address 127.3.15.130:42475)
0807 20:12:04.435052 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.435057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.435067 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:04.435106 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.435108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.435123 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.435197 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:04.435310 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.436524 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.437697 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.438432 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.438507 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.438597 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.438860 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.438900 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
I20260807 20:12:04.438985 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.434484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37730 (local address 127.3.15.129:41023)
0807 20:12:04.434790 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.434794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.434806 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:04.434941 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.434945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.434962 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.435042 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:04.435380 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.436494 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.437731 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.438435 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.438507 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.438625 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.438860 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.438901 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":102,"threads_started":1}
I20260807 20:12:04.439440 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.434383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:41023 (local address 127.0.0.1:37730)
0807 20:12:04.434700 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.434743 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.434809 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.435044 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.435047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.435049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:04.435320 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.435322 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.436576 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.436577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.437673 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.437674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.438656 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.438657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.438705 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.438709 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.438762 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.438815 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":5,"thread_start_us":58,"threads_started":1}
I20260807 20:12:04.439606 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.434588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:42475 (local address 127.0.0.1:38768)
0807 20:12:04.434853 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.434862 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.434906 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.435198 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.435199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.435201 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:04.435251 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.435253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.436632 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.436633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.437599 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.437601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.438639 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.438640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.438704 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.438709 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.438762 (+ 53us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.438814 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":5,"thread_start_us":103,"threads_started":1}
I20260807 20:12:04.440133 22944 tablet_service.cc:1467] Processing CreateTablet for tablet 8d90236c3b5c48d38afa92b603035f19 (DEFAULT_TABLE table=test-table [id=f5ddee65b39c4ca881d61bba6c7b2a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:04.440081 23226 tablet_service.cc:1467] Processing CreateTablet for tablet 8d90236c3b5c48d38afa92b603035f19 (DEFAULT_TABLE table=test-table [id=f5ddee65b39c4ca881d61bba6c7b2a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:04.440241 23086 tablet_service.cc:1467] Processing CreateTablet for tablet 8d90236c3b5c48d38afa92b603035f19 (DEFAULT_TABLE table=test-table [id=f5ddee65b39c4ca881d61bba6c7b2a45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:04.440505 22944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d90236c3b5c48d38afa92b603035f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:04.440507 23226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d90236c3b5c48d38afa92b603035f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:04.440528 23086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d90236c3b5c48d38afa92b603035f19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:04.443243 23315 tablet_bootstrap.cc:492] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Bootstrap starting.
I20260807 20:12:04.443895 23316 tablet_bootstrap.cc:492] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Bootstrap starting.
I20260807 20:12:04.444190 23317 tablet_bootstrap.cc:492] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Bootstrap starting.
I20260807 20:12:04.444396 23315 tablet_bootstrap.cc:654] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:04.444571 23316 tablet_bootstrap.cc:654] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:04.444821 23315 log.cc:824] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:04.444941 23316 log.cc:824] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:04.445045 23317 tablet_bootstrap.cc:654] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:04.445358 23317 log.cc:824] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:04.445617 23316 tablet_bootstrap.cc:492] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: No bootstrap required, opened a new log
I20260807 20:12:04.445636 23315 tablet_bootstrap.cc:492] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: No bootstrap required, opened a new log
I20260807 20:12:04.445685 23316 ts_tablet_manager.cc:1397] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 20:12:04.445698 23315 ts_tablet_manager.cc:1397] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:12:04.446154 23317 tablet_bootstrap.cc:492] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: No bootstrap required, opened a new log
I20260807 20:12:04.446219 23317 ts_tablet_manager.cc:1397] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:04.447856 23316 raft_consensus.cc:348] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.448005 23316 raft_consensus.cc:374] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:04.448041 23316 raft_consensus.cc:729] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e720622c3234a29806bb1dcf352b77e, State: Initialized, Role: FOLLOWER
I20260807 20:12:04.448168 23316 consensus_queue.cc:260] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.448472 23316 ts_tablet_manager.cc:1428] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 20:12:04.448534 23301 heartbeater.cc:503] Master 127.3.15.190:44973 was elected leader, sending a full tablet report...
I20260807 20:12:04.448405 23315 raft_consensus.cc:348] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.448977 23315 raft_consensus.cc:374] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:04.448951 23317 raft_consensus.cc:348] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.449101 23317 raft_consensus.cc:374] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:04.449142 23317 raft_consensus.cc:729] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801a2564aac446df8d937e927c8bd565, State: Initialized, Role: FOLLOWER
I20260807 20:12:04.449072 23315 raft_consensus.cc:729] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2913ef9f994a8faae10475ec5f9f69, State: Initialized, Role: FOLLOWER
I20260807 20:12:04.449287 23317 consensus_queue.cc:260] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.449332 23315 consensus_queue.cc:260] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.449555 23315 ts_tablet_manager.cc:1428] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 20:12:04.449566 23317 ts_tablet_manager.cc:1428] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 20:12:04.449616 23157 heartbeater.cc:503] Master 127.3.15.190:44973 was elected leader, sending a full tablet report...
I20260807 20:12:04.449669 23017 heartbeater.cc:503] Master 127.3.15.190:44973 was elected leader, sending a full tablet report...
W20260807 20:12:04.518026 23158 tablet.cc:2369] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:04.632412 23018 tablet.cc:2369] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:04.658167 23302 tablet.cc:2369] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:04.694067 23322 raft_consensus.cc:482] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:04.694254 23322 raft_consensus.cc:504] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.694690 23322 leader_election.cc:288] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da2913ef9f994a8faae10475ec5f9f69 (127.3.15.129:41023), 7e720622c3234a29806bb1dcf352b77e (127.3.15.131:36991)
I20260807 20:12:04.697502 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.694871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:41023 (local address 127.3.15.130:44125)
0807 20:12:04.694931 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.694943 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.694992 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.695192 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.695193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.695196 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:04.695271 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.695273 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.695918 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.695919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.696676 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.696678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.697253 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.697254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.697321 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.697324 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.697388 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.697431 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":5}
I20260807 20:12:04.697500 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.694932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:44125 (local address 127.3.15.129:41023)
0807 20:12:04.694976 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.694979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.694989 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:04.695100 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.695101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.695110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.695171 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:04.695343 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.695824 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.696768 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.697158 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.697243 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.697251 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.697395 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.697432 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 20:12:04.697788 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.694943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:36991 (local address 127.3.15.130:60193)
0807 20:12:04.695137 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.695146 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.695228 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.695409 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:04.695410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:04.695412 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:04.695488 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.695490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.696126 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.696127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.696957 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:04.696959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.697539 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.697540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:04.697619 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.697623 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:04.697674 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:12:04.697711 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":133,"threads_started":1}
I20260807 20:12:04.697854 22966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d90236c3b5c48d38afa92b603035f19" candidate_uuid: "801a2564aac446df8d937e927c8bd565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da2913ef9f994a8faae10475ec5f9f69" is_pre_election: true
I20260807 20:12:04.697938 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.694973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:60193 (local address 127.3.15.131:36991)
0807 20:12:04.695032 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.695034 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.695213 (+ 179us) server_negotiation.cc:407] Connection header received
0807 20:12:04.695297 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.695298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.695311 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.695384 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:04.695567 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.696040 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.697044 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.697455 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.697538 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.697546 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.697761 (+ 215us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.697805 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:12:04.698055 22966 raft_consensus.cc:2454] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801a2564aac446df8d937e927c8bd565 in term 0.
I20260807 20:12:04.698256 23248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d90236c3b5c48d38afa92b603035f19" candidate_uuid: "801a2564aac446df8d937e927c8bd565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e720622c3234a29806bb1dcf352b77e" is_pre_election: true
I20260807 20:12:04.698316 23036 leader_election.cc:302] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801a2564aac446df8d937e927c8bd565, da2913ef9f994a8faae10475ec5f9f69; no voters:
I20260807 20:12:04.698387 23248 raft_consensus.cc:2454] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 801a2564aac446df8d937e927c8bd565 in term 0.
I20260807 20:12:04.698443 23322 raft_consensus.cc:2781] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:04.698494 23322 raft_consensus.cc:482] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:04.698524 23322 raft_consensus.cc:3037] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:04.699366 23322 raft_consensus.cc:504] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.699503 23322 leader_election.cc:288] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [CANDIDATE]: Term 1 election: Requested vote from peers da2913ef9f994a8faae10475ec5f9f69 (127.3.15.129:41023), 7e720622c3234a29806bb1dcf352b77e (127.3.15.131:36991)
I20260807 20:12:04.699659 22966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d90236c3b5c48d38afa92b603035f19" candidate_uuid: "801a2564aac446df8d937e927c8bd565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da2913ef9f994a8faae10475ec5f9f69"
I20260807 20:12:04.699695 23248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d90236c3b5c48d38afa92b603035f19" candidate_uuid: "801a2564aac446df8d937e927c8bd565" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e720622c3234a29806bb1dcf352b77e"
I20260807 20:12:04.699743 22966 raft_consensus.cc:3037] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:04.699790 23248 raft_consensus.cc:3037] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:04.700482 22966 raft_consensus.cc:2454] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801a2564aac446df8d937e927c8bd565 in term 1.
I20260807 20:12:04.700594 23248 raft_consensus.cc:2454] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 801a2564aac446df8d937e927c8bd565 in term 1.
I20260807 20:12:04.700737 23036 leader_election.cc:302] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 801a2564aac446df8d937e927c8bd565, da2913ef9f994a8faae10475ec5f9f69; no voters:
I20260807 20:12:04.700856 23322 raft_consensus.cc:2781] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:04.701017 23322 raft_consensus.cc:686] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 1 LEADER]: Becoming Leader. State: Replica: 801a2564aac446df8d937e927c8bd565, State: Running, Role: LEADER
I20260807 20:12:04.701133 23322 consensus_queue.cc:237] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } }
I20260807 20:12:04.701969 22815 catalog_manager.cc:5495] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 reported cstate change: term changed from 0 to 1, leader changed from <none> to 801a2564aac446df8d937e927c8bd565 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "801a2564aac446df8d937e927c8bd565" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801a2564aac446df8d937e927c8bd565" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 42475 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:04.725675 23314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:04.723159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.3.15.130:42475)
0807 20:12:04.723231 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:04.723233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:04.723241 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:04.723276 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:04.723277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:04.723298 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:04.723361 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:04.723489 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.723952 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.724698 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:04.724894 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:04.724944 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:04.725115 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:04.725248 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:04.725348 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:04.725476 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:12:04.725489 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 20:12:04.727685 22966 raft_consensus.cc:1261] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 1 FOLLOWER]: Refusing update from remote peer 801a2564aac446df8d937e927c8bd565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:04.727712 23248 raft_consensus.cc:1261] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 1 FOLLOWER]: Refusing update from remote peer 801a2564aac446df8d937e927c8bd565: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:04.728103 23326 consensus_queue.cc:1035] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e720622c3234a29806bb1dcf352b77e" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 36991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:04.728232 23322 consensus_queue.cc:1035] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da2913ef9f994a8faae10475ec5f9f69" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:04.731256 23330 mvcc.cc:205] Tried to move back new op lower bound from 7316002917281517568 to 7316002917176479744. Current Snapshot: MvccSnapshot[applied={T|T < 7316002917281517568}]
I20260807 20:12:04.731381 23331 mvcc.cc:205] Tried to move back new op lower bound from 7316002917281517568 to 7316002917176479744. Current Snapshot: MvccSnapshot[applied={T|T < 7316002917281517568 or (T in {7316002917281517568})}]
I20260807 20:12:04.732511 22815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:04.732623 22815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:04.733399 22815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80aebf2a866c46e49f32383bd6b8a6ad: Sending DeleteTablet for 3 replicas of tablet 8d90236c3b5c48d38afa92b603035f19
I20260807 20:12:04.733762 23226 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d90236c3b5c48d38afa92b603035f19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:04 UTC) from {username='slave'} at 127.0.0.1:55214
I20260807 20:12:04.733808 22944 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d90236c3b5c48d38afa92b603035f19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:04 UTC) from {username='slave'} at 127.0.0.1:37730
I20260807 20:12:04.734014 23339 tablet_replica.cc:330] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: stopping tablet replica
I20260807 20:12:04.734015 23086 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d90236c3b5c48d38afa92b603035f19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:04 UTC) from {username='slave'} at 127.0.0.1:38768
I20260807 20:12:04.734133 23339 raft_consensus.cc:2229] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:04.734257 23339 raft_consensus.cc:2258] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:04.734261 23340 tablet_replica.cc:330] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: stopping tablet replica
I20260807 20:12:04.734422 23340 raft_consensus.cc:2229] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:04.734483 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22881
I20260807 20:12:04.734586 23340 raft_consensus.cc:2258] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:04.734854 23339 ts_tablet_manager.cc:1905] T 8d90236c3b5c48d38afa92b603035f19 P da2913ef9f994a8faae10475ec5f9f69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:04.737999 23340 ts_tablet_manager.cc:1905] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:04.739749 23340 ts_tablet_manager.cc:1918] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:04.739830 23340 log.cc:1196] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/8d90236c3b5c48d38afa92b603035f19
I20260807 20:12:04.740120 23340 ts_tablet_manager.cc:1939] T 8d90236c3b5c48d38afa92b603035f19 P 801a2564aac446df8d937e927c8bd565: Deleting consensus metadata
I20260807 20:12:04.740537 22800 catalog_manager.cc:4841] TS 801a2564aac446df8d937e927c8bd565 (127.3.15.130:42475): tablet 8d90236c3b5c48d38afa92b603035f19 (table test-table [id=f5ddee65b39c4ca881d61bba6c7b2a45]) successfully deleted
I20260807 20:12:04.740973 23338 tablet_replica.cc:330] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: stopping tablet replica
I20260807 20:12:04.741096 23338 raft_consensus.cc:2229] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:04.741211 23338 raft_consensus.cc:2258] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:04.741647 23338 ts_tablet_manager.cc:1905] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 20:12:04.742182 22800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:41023 (error 108)
W20260807 20:12:04.742800 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:04.742558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:41023 (local address 127.0.0.1:37746)
0807 20:12:04.742623 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:04.742635 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:04.742676 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:04.742744 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:41023: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 20:12:04.742913 22800 catalog_manager.cc:4584] TS da2913ef9f994a8faae10475ec5f9f69 (127.3.15.129:41023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d90236c3b5c48d38afa92b603035f19: Network error: Client connection negotiation failed: client connection to 127.3.15.129:41023: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260807 20:12:04.743239 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23020
I20260807 20:12:04.743261 23338 ts_tablet_manager.cc:1918] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:04.743324 23338 log.cc:1196] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/8d90236c3b5c48d38afa92b603035f19
I20260807 20:12:04.743577 23338 ts_tablet_manager.cc:1939] T 8d90236c3b5c48d38afa92b603035f19 P 7e720622c3234a29806bb1dcf352b77e: Deleting consensus metadata
I20260807 20:12:04.744273 22799 catalog_manager.cc:4841] TS 7e720622c3234a29806bb1dcf352b77e (127.3.15.131:36991): tablet 8d90236c3b5c48d38afa92b603035f19 (table test-table [id=f5ddee65b39c4ca881d61bba6c7b2a45]) successfully deleted
I20260807 20:12:04.751259 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23160
I20260807 20:12:04.756633 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 22784
2026-08-07T20:12:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (992 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:04 dist-test-slave-1x4g krb5kdc[23345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:04 dist-test-slave-1x4g krb5kdc[23345](info): set up 2 sockets
Aug 07 20:12:04 dist-test-slave-1x4g krb5kdc[23345](info): commencing operation
krb5kdc: starting...
W20260807 20:12:05.833328 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:05 dist-test-slave-1x4g krb5kdc[23345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:05.959933 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39151
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:06.042402 23359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:06.042641 23359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:06.042680 23359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:06.043812 23359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:06.043880 23359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:06.043905 23359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:06.043933 23359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:06.043964 23359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:06.045482 23359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:06.045756 23359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:06.045991 23359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:06.050971 23366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.050982 23369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.051023 23365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.051098 23359 server_base.cc:1034] running on GCE node
I20260807 20:12:06.051479 23359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:06.051790 23359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:06.052954 23359 hybrid_clock.cc:648] HybridClock initialized: now 1786133526052934 us; error 37 us; skew 500 ppm
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:06.055418 23359 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:06.055876 23359 webserver.cc:460] Webserver started at http://127.3.15.190:36249/ using document root <none> and password file <none>
I20260807 20:12:06.056078 23359 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:06.056119 23359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:06.056207 23359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:06.057266 23359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "e0c26fafbad342268913ea8e40ed9111"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "9433b485a46358808aa34cb78517c6d5"
server_key_iv: "be7419925a8198efb14ee762b45b1f77"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.057575 23359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "e0c26fafbad342268913ea8e40ed9111"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "9433b485a46358808aa34cb78517c6d5"
server_key_iv: "be7419925a8198efb14ee762b45b1f77"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.058843 23359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:06.059532 23376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.059736 23359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:06.059823 23359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "e0c26fafbad342268913ea8e40ed9111"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "9433b485a46358808aa34cb78517c6d5"
server_key_iv: "be7419925a8198efb14ee762b45b1f77"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.059877 23359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:06.071533 23359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:06.073904 23359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:06.074030 23359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:06.078065 23359 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39151
I20260807 20:12:06.078153 23429 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39151 every 8 connection(s)
I20260807 20:12:06.078500 23359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:06.079433 23430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:06.081830 23430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: Bootstrap starting.
I20260807 20:12:06.082420 23430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:06.082736 23430 log.cc:824] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:06.083441 23430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: No bootstrap required, opened a new log
I20260807 20:12:06.084988 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23359
I20260807 20:12:06.085112 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:06.085314 3134 external_mini_cluster.cc:1428] Setting key be199eaf8e4972aaa089669daf3decff
I20260807 20:12:06.085244 23430 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26fafbad342268913ea8e40ed9111" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39151 } }
I20260807 20:12:06.085423 23430 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:06.085470 23430 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0c26fafbad342268913ea8e40ed9111, State: Initialized, Role: FOLLOWER
I20260807 20:12:06.085635 23430 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26fafbad342268913ea8e40ed9111" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39151 } }
I20260807 20:12:06.085734 23430 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:06.085778 23430 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:06.085834 23430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:06.086585 23430 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26fafbad342268913ea8e40ed9111" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39151 } }
I20260807 20:12:06.086702 23430 leader_election.cc:302] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0c26fafbad342268913ea8e40ed9111; no voters:
I20260807 20:12:06.086860 23430 leader_election.cc:288] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:06.086887 23435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:06.087001 23435 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [term 1 LEADER]: Becoming Leader. State: Replica: e0c26fafbad342268913ea8e40ed9111, State: Running, Role: LEADER
I20260807 20:12:06.087095 23435 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26fafbad342268913ea8e40ed9111" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39151 } }
I20260807 20:12:06.087133 23430 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:06.087423 23436 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0c26fafbad342268913ea8e40ed9111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26fafbad342268913ea8e40ed9111" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39151 } } }
I20260807 20:12:06.087433 23438 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0c26fafbad342268913ea8e40ed9111. Latest consensus state: current_term: 1 leader_uuid: "e0c26fafbad342268913ea8e40ed9111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0c26fafbad342268913ea8e40ed9111" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39151 } } }
I20260807 20:12:06.087649 23436 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:06.087670 23438 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:06.089370 23453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:06.089430 23453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:06.089484 23444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:06.089632 23444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:06.091140 23444 catalog_manager.cc:1347] Generated new cluster ID: b0654d591313457d8ca632c62ed861b2
I20260807 20:12:06.091192 23444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133525, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:06.098492 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.085820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35622 (local address 127.3.15.190:39151)
0807 20:12:06.086014 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.086024 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.086069 (+ 45us) server_negotiation.cc:407] Connection header received
0807 20:12:06.086492 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.086501 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.086566 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.086728 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:06.087541 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.088951 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.089999 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.090637 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.090747 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.093920 (+ 3173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:06.093924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:06.093925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:06.093938 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:06.096028 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.096456 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.096460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.096460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.097233 (+ 773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.097477 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.097478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.097479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.097843 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:06.098038 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.098115 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.098274 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260807 20:12:06.123497 23444 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:06.124300 23444 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:06.131278 23444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: Generated new TSK 0
I20260807 20:12:06.131522 23444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.15.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
Entry for principal kudu/127.3.15.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab.
I20260807 20:12:06.159793 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39151
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:06.243314 23458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:06.243518 23458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:06.243542 23458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:06.244721 23458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:06.244766 23458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:06.244858 23458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:12:06.246305 23458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.129.keytab
--principal=kudu/127.3.15.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:06.246503 23458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:06.246692 23458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:06.251878 23464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.251884 23463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.251878 23466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.251933 23458 server_base.cc:1034] running on GCE node
I20260807 20:12:06.252321 23458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:06.252529 23458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:06.253671 23458 hybrid_clock.cc:648] HybridClock initialized: now 1786133526253643 us; error 44 us; skew 500 ppm
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:06.256356 23458 init.cc:377] Logged in from keytab as kudu/127.3.15.129@KRBTEST.COM (short username kudu)
I20260807 20:12:06.256819 23458 webserver.cc:460] Webserver started at http://127.3.15.129:46463/ using document root <none> and password file <none>
I20260807 20:12:06.257052 23458 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:06.257094 23458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:06.257192 23458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:06.258221 23458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "df7daaae7df24e7baa5cf73ca15f6ab7"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "2f303c97e77565faa84fabd5bb51c28f"
server_key_iv: "5b002a7c12e0bbf8947a69ce86240272"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.258536 23458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "df7daaae7df24e7baa5cf73ca15f6ab7"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "2f303c97e77565faa84fabd5bb51c28f"
server_key_iv: "5b002a7c12e0bbf8947a69ce86240272"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.259820 23458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:06.260524 23475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.260710 23458 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:06.260838 23458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "df7daaae7df24e7baa5cf73ca15f6ab7"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "2f303c97e77565faa84fabd5bb51c28f"
server_key_iv: "5b002a7c12e0bbf8947a69ce86240272"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.260919 23458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:06.273319 23458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:06.275271 23458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:06.275442 23458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:06.275712 23458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:06.276042 23458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:06.276096 23458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.276132 23458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:06.276161 23458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.282274 23458 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:41519
I20260807 20:12:06.282325 23597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:41519 every 8 connection(s)
I20260807 20:12:06.282666 23458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:06.285784 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23458
I20260807 20:12:06.285898 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:06.286091 3134 external_mini_cluster.cc:1428] Setting key 051a16bdcd5f4fd0826581ff917be8a5
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.129@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:06.292706 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.283365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:43357 (local address 127.3.15.190:39151)
0807 20:12:06.283437 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.283439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.283679 (+ 240us) server_negotiation.cc:407] Connection header received
0807 20:12:06.284192 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.284194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.284206 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.284266 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:06.285331 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.285881 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.286970 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.287260 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.287331 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.289714 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:06.289715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:06.289716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:06.289724 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:06.291199 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.291629 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.291630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.291631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.291697 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.291960 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.291961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.291961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.292086 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:06.292157 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.292496 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.292614 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:12:06.292903 23600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.283333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39151 (local address 127.3.15.129:43357)
0807 20:12:06.283520 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.283571 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.284099 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.284347 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.284350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.285038 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:06.285245 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.285251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.285987 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.285988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.286866 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.286869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.287356 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.287357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.287487 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.288066 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:06.288075 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:06.289606 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:06.291305 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:06.291313 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:06.291315 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:06.291549 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:06.291772 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:06.291773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:06.291773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:06.291890 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:06.292161 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:06.292169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:06.292347 (+ 178us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.292469 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.292638 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":112,"threads_started":1}
I20260807 20:12:06.293417 23598 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39151
I20260807 20:12:06.293515 23598 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:06.293716 23598 heartbeater.cc:511] Master 127.3.15.190:39151 requested a full tablet report, sending...
I20260807 20:12:06.294279 23393 ts_manager.cc:194] Registered new tserver with Master: df7daaae7df24e7baa5cf73ca15f6ab7 (127.3.15.129:41519)
I20260807 20:12:06.294898 23393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.129@KRBTEST.COM'} at 127.3.15.129:43357
WARNING: no policy specified for kudu/127.3.15.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
Entry for principal kudu/127.3.15.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab.
I20260807 20:12:06.313971 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39151
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:06.400101 23603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:06.400327 23603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:06.400364 23603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:06.401518 23603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:06.401690 23603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:06.401788 23603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:12:06.403577 23603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.130.keytab
--principal=kudu/127.3.15.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:06.403878 23603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:06.404162 23603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:06.409389 23608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.409407 23603 server_base.cc:1034] running on GCE node
W20260807 20:12:06.409389 23611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.409418 23609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.409838 23603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:06.410111 23603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:06.411276 23603 hybrid_clock.cc:648] HybridClock initialized: now 1786133526411268 us; error 40 us; skew 500 ppm
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:06.413846 23603 init.cc:377] Logged in from keytab as kudu/127.3.15.130@KRBTEST.COM (short username kudu)
I20260807 20:12:06.414312 23603 webserver.cc:460] Webserver started at http://127.3.15.130:40689/ using document root <none> and password file <none>
I20260807 20:12:06.414541 23603 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:06.414614 23603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:06.414767 23603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:06.415802 23603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "8d990c478e104ee28df41c4c56fe57b4"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "954d8fe17d3b73d4d2f1520f95ce1be9"
server_key_iv: "150db4476f726ee7d8840b8942134ee5"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.416162 23603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "8d990c478e104ee28df41c4c56fe57b4"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "954d8fe17d3b73d4d2f1520f95ce1be9"
server_key_iv: "150db4476f726ee7d8840b8942134ee5"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.417644 23603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:06.418421 23620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.418614 23603 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:06.418722 23603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "8d990c478e104ee28df41c4c56fe57b4"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "954d8fe17d3b73d4d2f1520f95ce1be9"
server_key_iv: "150db4476f726ee7d8840b8942134ee5"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.418793 23603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:06.426092 23603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:06.428303 23603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:06.428459 23603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:06.428685 23603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:06.429018 23603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:06.429076 23603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.429118 23603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:06.429147 23603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.434957 23603 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:43863
I20260807 20:12:06.435010 23737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:43863 every 8 connection(s)
I20260807 20:12:06.435305 23603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:06.439687 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23603
I20260807 20:12:06.439817 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:06.440006 3134 external_mini_cluster.cc:1428] Setting key bf67a5cb571159fef8db7825bfe431c3
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.130@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:06.445237 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.435984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:50103 (local address 127.3.15.190:39151)
0807 20:12:06.436036 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.436039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.436247 (+ 208us) server_negotiation.cc:407] Connection header received
0807 20:12:06.436714 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.436715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.436726 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.436806 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:06.437707 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.438205 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.439149 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.439331 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.439386 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.442139 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:06.442140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:06.442140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:06.442147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:06.443522 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.444055 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.444056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.444057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.444119 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.444489 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.444490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.444490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.444630 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:06.444709 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.445041 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.445135 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:06.445461 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.435975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39151 (local address 127.3.15.130:50103)
0807 20:12:06.436128 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.436162 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.436624 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.436884 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.436888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.437433 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:06.437625 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.437630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.438284 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.438285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.439073 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.439075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.439406 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.439407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.439509 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.440031 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:06.440039 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:06.442031 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:06.443623 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:06.443630 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:06.443633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:06.443951 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:06.444281 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:06.444282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:06.444282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:06.444410 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:06.444713 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:06.444721 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:06.444903 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.445018 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.445212 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":93,"threads_started":1}
I20260807 20:12:06.446079 23738 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39151
I20260807 20:12:06.446189 23738 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:06.446406 23738 heartbeater.cc:511] Master 127.3.15.190:39151 requested a full tablet report, sending...
I20260807 20:12:06.446805 23393 ts_manager.cc:194] Registered new tserver with Master: 8d990c478e104ee28df41c4c56fe57b4 (127.3.15.130:43863)
I20260807 20:12:06.447198 23393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.130@KRBTEST.COM'} at 127.3.15.130:50103
WARNING: no policy specified for kudu/127.3.15.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
Entry for principal kudu/127.3.15.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab.
I20260807 20:12:06.467305 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39151
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:06.551918 23743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:06.552129 23743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:06.552153 23743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:06.553432 23743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:06.553476 23743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:06.553503 23743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:12:06.554903 23743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.131.keytab
--principal=kudu/127.3.15.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:06.555104 23743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:06.555280 23743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:06.560485 23750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.560474 23749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.560474 23752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.560811 23743 server_base.cc:1034] running on GCE node
I20260807 20:12:06.561023 23743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:06.561260 23743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:06.562428 23743 hybrid_clock.cc:648] HybridClock initialized: now 1786133526562423 us; error 37 us; skew 500 ppm
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:06.565282 23743 init.cc:377] Logged in from keytab as kudu/127.3.15.131@KRBTEST.COM (short username kudu)
I20260807 20:12:06.565788 23743 webserver.cc:460] Webserver started at http://127.3.15.131:38767/ using document root <none> and password file <none>
I20260807 20:12:06.566036 23743 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:06.566110 23743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:06.566263 23743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:06.567374 23743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "3734ca1596d1492abe8f81c1ffc3a02b"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "81786ea1c44471855af820fa92db1ef0"
server_key_iv: "3cc140c9b506be52b307c49e387af755"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.567741 23743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "3734ca1596d1492abe8f81c1ffc3a02b"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "81786ea1c44471855af820fa92db1ef0"
server_key_iv: "3cc140c9b506be52b307c49e387af755"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.569205 23743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:06.569994 23761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.570169 23743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:06.570272 23743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "3734ca1596d1492abe8f81c1ffc3a02b"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "81786ea1c44471855af820fa92db1ef0"
server_key_iv: "3cc140c9b506be52b307c49e387af755"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.570364 23743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:06.586828 23743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:06.589118 23743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:06.589274 23743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:06.589501 23743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:06.589819 23743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:06.589881 23743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.589923 23743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:06.589952 23743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.595717 23743 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:33089
I20260807 20:12:06.595793 23880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:33089 every 8 connection(s)
I20260807 20:12:06.596129 23743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
Aug 07 20:12:06 dist-test-slave-1x4g krb5kdc[23345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133526, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.131@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:06.602723 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23743
I20260807 20:12:06.602840 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:06.603040 3134 external_mini_cluster.cc:1428] Setting key ab52448bee6e5baf70d20ad0b8f134da
I20260807 20:12:06.606081 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.596843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:48629 (local address 127.3.15.190:39151)
0807 20:12:06.596885 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.596888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.597113 (+ 225us) server_negotiation.cc:407] Connection header received
0807 20:12:06.597655 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.597657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.597669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.597733 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:06.598814 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.599327 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.600348 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.600562 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.600644 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.602922 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:06.602924 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:06.602924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:06.602933 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:06.604523 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.605055 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.605056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.605057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.605125 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.605391 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.605392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.605392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.605518 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:06.605590 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.605909 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.605989 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:12:06.606279 23883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.596851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39151 (local address 127.3.15.131:48629)
0807 20:12:06.596994 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.597029 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.597544 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.597809 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.597813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.598414 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:06.598726 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.598733 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.599434 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.599436 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.600254 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.600256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.600650 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.600652 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.600822 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.601356 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:06.601365 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:06.602809 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:06.604638 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:06.604645 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:06.604647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:06.604964 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:06.605203 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:06.605204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:06.605204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:06.605316 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:06.605592 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:06.605600 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:06.605765 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.605868 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.606029 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260807 20:12:06.606752 23881 heartbeater.cc:347] Connected to a master server at 127.3.15.190:39151
I20260807 20:12:06.606854 23881 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:06.607038 23881 heartbeater.cc:511] Master 127.3.15.190:39151 requested a full tablet report, sending...
I20260807 20:12:06.607410 23393 ts_manager.cc:194] Registered new tserver with Master: 3734ca1596d1492abe8f81c1ffc3a02b (127.3.15.131:33089)
I20260807 20:12:06.607785 23393 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.15.131@KRBTEST.COM'} at 127.3.15.131:48629
I20260807 20:12:06.614872 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:06.621789 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.616191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35624 (local address 127.3.15.190:39151)
0807 20:12:06.616263 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.616265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.616348 (+ 83us) server_negotiation.cc:407] Connection header received
0807 20:12:06.616389 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.616390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.616399 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.616441 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:06.617050 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.617522 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.618211 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.618389 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.618439 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.619523 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:06.619524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:06.619524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:06.619530 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:06.620724 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.621078 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.621079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.621079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.621141 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:06.621307 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:06.621308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:06.621308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:06.621449 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:06.621523 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.621660 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.621724 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 20:12:06.623502 23393 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35624:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:06.625173 23393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:06.631520 23891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.627346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:33089 (local address 127.0.0.1:41634)
0807 20:12:06.627481 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.627492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.627557 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.627876 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.627879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.627881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:06.627950 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.627952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.629316 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.629318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.630409 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.630411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.631190 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.631191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.631272 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.631289 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.631356 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.631410 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":5,"thread_start_us":55,"threads_started":1}
I20260807 20:12:06.631628 23889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.627300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:41519 (local address 127.0.0.1:36576)
0807 20:12:06.627445 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.627492 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.627557 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.628151 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.628153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.628155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:06.628210 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.628213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.629285 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.629286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.630563 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.630566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.631370 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.631371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.631437 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.631441 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.631505 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.631543 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":55,"thread_start_us":95,"threads_started":1}
I20260807 20:12:06.631682 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.627681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.3.15.129:41519)
0807 20:12:06.627880 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.627884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.627894 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:06.627934 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.627936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.627951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.628027 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:06.628293 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.629178 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.630662 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.631278 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.631356 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.631416 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.631532 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.631585 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":84,"threads_started":1}
I20260807 20:12:06.631968 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.627467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.3.15.131:33089)
0807 20:12:06.627679 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.627684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.627695 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:12:06.627746 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.627748 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.627766 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.627857 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:06.628034 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.629195 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.630493 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.631099 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.631171 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.631227 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.631723 (+ 496us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.631887 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
I20260807 20:12:06.632373 23526 tablet_service.cc:1467] Processing CreateTablet for tablet bf18db2cac8b4ca88a806e513e331788 (DEFAULT_TABLE table=test-table [id=8ca94f3bca1942f29cff553ca10f22da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:06.632555 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.627543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:43863 (local address 127.0.0.1:56560)
0807 20:12:06.628124 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.628135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.628184 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.628457 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.628458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.628460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:06.628505 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.628507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.629727 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.629728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.630771 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.630774 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.632209 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.632210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.632272 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.632278 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.632449 (+ 171us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.632490 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":1,"thread_start_us":45,"threads_started":1}
I20260807 20:12:06.632654 23811 tablet_service.cc:1467] Processing CreateTablet for tablet bf18db2cac8b4ca88a806e513e331788 (DEFAULT_TABLE table=test-table [id=8ca94f3bca1942f29cff553ca10f22da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:06.632822 23526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf18db2cac8b4ca88a806e513e331788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:06.633013 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.627943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56560 (local address 127.3.15.130:43863)
0807 20:12:06.628300 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.628304 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.628313 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:06.628350 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.628352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.628366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.628443 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:06.628566 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.629623 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.630854 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.631965 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.632060 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.632128 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.632878 (+ 750us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.632925 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":85,"threads_started":1}
I20260807 20:12:06.633028 23811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf18db2cac8b4ca88a806e513e331788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:06.633659 23670 tablet_service.cc:1467] Processing CreateTablet for tablet bf18db2cac8b4ca88a806e513e331788 (DEFAULT_TABLE table=test-table [id=8ca94f3bca1942f29cff553ca10f22da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:06.633987 23670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf18db2cac8b4ca88a806e513e331788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:06.635762 23896 tablet_bootstrap.cc:492] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Bootstrap starting.
I20260807 20:12:06.636267 23897 tablet_bootstrap.cc:492] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: Bootstrap starting.
I20260807 20:12:06.636479 23896 tablet_bootstrap.cc:654] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:06.637054 23897 tablet_bootstrap.cc:654] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:06.637145 23898 tablet_bootstrap.cc:492] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Bootstrap starting.
I20260807 20:12:06.637174 23896 log.cc:824] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:06.637619 23897 log.cc:824] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:06.637773 23898 tablet_bootstrap.cc:654] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:06.638023 23896 tablet_bootstrap.cc:492] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: No bootstrap required, opened a new log
I20260807 20:12:06.638113 23896 ts_tablet_manager.cc:1397] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:06.638381 23898 log.cc:824] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:06.639122 23898 tablet_bootstrap.cc:492] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: No bootstrap required, opened a new log
I20260807 20:12:06.639134 23897 tablet_bootstrap.cc:492] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: No bootstrap required, opened a new log
I20260807 20:12:06.639204 23897 ts_tablet_manager.cc:1397] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 20:12:06.639207 23898 ts_tablet_manager.cc:1397] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:06.640219 23896 raft_consensus.cc:348] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.640401 23896 raft_consensus.cc:374] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:06.640439 23896 raft_consensus.cc:729] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3734ca1596d1492abe8f81c1ffc3a02b, State: Initialized, Role: FOLLOWER
I20260807 20:12:06.640583 23896 consensus_queue.cc:260] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.640638 23898 raft_consensus.cc:348] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.640852 23898 raft_consensus.cc:374] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:06.640892 23896 ts_tablet_manager.cc:1428] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:06.640901 23898 raft_consensus.cc:729] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d990c478e104ee28df41c4c56fe57b4, State: Initialized, Role: FOLLOWER
I20260807 20:12:06.641019 23881 heartbeater.cc:503] Master 127.3.15.190:39151 was elected leader, sending a full tablet report...
I20260807 20:12:06.641019 23898 consensus_queue.cc:260] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.641366 23898 ts_tablet_manager.cc:1428] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 20:12:06.641247 23897 raft_consensus.cc:348] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.641420 23897 raft_consensus.cc:374] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:06.641465 23897 raft_consensus.cc:729] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df7daaae7df24e7baa5cf73ca15f6ab7, State: Initialized, Role: FOLLOWER
I20260807 20:12:06.641572 23897 consensus_queue.cc:260] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.641638 23738 heartbeater.cc:503] Master 127.3.15.190:39151 was elected leader, sending a full tablet report...
I20260807 20:12:06.641834 23897 ts_tablet_manager.cc:1428] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:06.641908 23598 heartbeater.cc:503] Master 127.3.15.190:39151 was elected leader, sending a full tablet report...
I20260807 20:12:06.663344 23903 raft_consensus.cc:482] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:06.663491 23903 raft_consensus.cc:504] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.663957 23903 leader_election.cc:288] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df7daaae7df24e7baa5cf73ca15f6ab7 (127.3.15.129:41519), 3734ca1596d1492abe8f81c1ffc3a02b (127.3.15.131:33089)
I20260807 20:12:06.666921 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.664342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:41705 (local address 127.3.15.129:41519)
0807 20:12:06.664399 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.664402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.664411 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:06.664440 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.664441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.664454 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.664522 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:06.664689 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.665233 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.666171 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.666555 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.666635 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.666645 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.666797 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.666842 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:12:06.666957 23892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.664199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:45451 (local address 127.3.15.131:33089)
0807 20:12:06.664271 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.664273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.664436 (+ 163us) server_negotiation.cc:407] Connection header received
0807 20:12:06.664493 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.664494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.664505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.664568 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:06.664743 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.665236 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.666167 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.666551 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.666645 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.666653 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.666846 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.666896 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 20:12:06.666944 23740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.664098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:41519 (local address 127.3.15.130:41705)
0807 20:12:06.664150 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.664164 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.664210 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.664540 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.664542 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.664544 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:06.664618 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.664620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.665324 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.665325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.666067 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.666069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.666638 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.666639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.666720 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.666723 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.666774 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.666828 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 20:12:06.666965 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.664145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:33089 (local address 127.3.15.130:45451)
0807 20:12:06.664349 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.664359 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:06.664429 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:06.664566 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:06.664566 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:06.664568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:06.664667 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.664669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.665328 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.665329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.666068 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:06.666069 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.666670 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.666671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:06.666745 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.666749 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 20:12:06.666823 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 20:12:06.666857 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":154,"threads_started":1}
I20260807 20:12:06.667279 23548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf18db2cac8b4ca88a806e513e331788" candidate_uuid: "8d990c478e104ee28df41c4c56fe57b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" is_pre_election: true
I20260807 20:12:06.667433 23548 raft_consensus.cc:2454] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d990c478e104ee28df41c4c56fe57b4 in term 0.
I20260807 20:12:06.667656 23624 leader_election.cc:302] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8d990c478e104ee28df41c4c56fe57b4, df7daaae7df24e7baa5cf73ca15f6ab7; no voters:
I20260807 20:12:06.667659 23831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf18db2cac8b4ca88a806e513e331788" candidate_uuid: "8d990c478e104ee28df41c4c56fe57b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" is_pre_election: true
I20260807 20:12:06.667802 23831 raft_consensus.cc:2454] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d990c478e104ee28df41c4c56fe57b4 in term 0.
I20260807 20:12:06.667804 23903 raft_consensus.cc:2781] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:06.667870 23903 raft_consensus.cc:482] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:06.667902 23903 raft_consensus.cc:3037] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:06.668656 23903 raft_consensus.cc:504] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.668824 23903 leader_election.cc:288] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [CANDIDATE]: Term 1 election: Requested vote from peers df7daaae7df24e7baa5cf73ca15f6ab7 (127.3.15.129:41519), 3734ca1596d1492abe8f81c1ffc3a02b (127.3.15.131:33089)
I20260807 20:12:06.668982 23548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf18db2cac8b4ca88a806e513e331788" candidate_uuid: "8d990c478e104ee28df41c4c56fe57b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7"
I20260807 20:12:06.669003 23831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf18db2cac8b4ca88a806e513e331788" candidate_uuid: "8d990c478e104ee28df41c4c56fe57b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3734ca1596d1492abe8f81c1ffc3a02b"
I20260807 20:12:06.669070 23831 raft_consensus.cc:3037] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:06.669070 23548 raft_consensus.cc:3037] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:06.669972 23831 raft_consensus.cc:2454] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d990c478e104ee28df41c4c56fe57b4 in term 1.
I20260807 20:12:06.669972 23548 raft_consensus.cc:2454] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d990c478e104ee28df41c4c56fe57b4 in term 1.
I20260807 20:12:06.670163 23623 leader_election.cc:302] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3734ca1596d1492abe8f81c1ffc3a02b, 8d990c478e104ee28df41c4c56fe57b4; no voters:
I20260807 20:12:06.670251 23903 raft_consensus.cc:2781] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:06.670397 23903 raft_consensus.cc:686] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 1 LEADER]: Becoming Leader. State: Replica: 8d990c478e104ee28df41c4c56fe57b4, State: Running, Role: LEADER
I20260807 20:12:06.670506 23903 consensus_queue.cc:237] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } }
I20260807 20:12:06.671325 23392 catalog_manager.cc:5495] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8d990c478e104ee28df41c4c56fe57b4 (127.3.15.130). New cstate: current_term: 1 leader_uuid: "8d990c478e104ee28df41c4c56fe57b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d990c478e104ee28df41c4c56fe57b4" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 43863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 } health_report { overall_health: UNKNOWN } } }
I20260807 20:12:06.676112 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.673959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.3.15.130:43863)
0807 20:12:06.674005 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.674007 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.674030 (+ 23us) server_negotiation.cc:407] Connection header received
0807 20:12:06.674118 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.674119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.674143 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.674218 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:06.674337 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.674770 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.675470 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.675648 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.675696 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.675819 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:06.675908 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:06.675966 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.676030 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.676045 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:06.677815 23831 raft_consensus.cc:1261] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 1 FOLLOWER]: Refusing update from remote peer 8d990c478e104ee28df41c4c56fe57b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:06.677843 23548 raft_consensus.cc:1261] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7 [term 1 FOLLOWER]: Refusing update from remote peer 8d990c478e104ee28df41c4c56fe57b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:06.678169 23903 consensus_queue.cc:1035] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3734ca1596d1492abe8f81c1ffc3a02b" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 33089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:06.678301 23907 consensus_queue.cc:1035] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df7daaae7df24e7baa5cf73ca15f6ab7" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:06.680495 23910 mvcc.cc:205] Tried to move back new op lower bound from 7316002925269897216 to 7316002925243092992. Current Snapshot: MvccSnapshot[applied={T|T < 7316002925269897216}]
I20260807 20:12:06.680836 23911 mvcc.cc:205] Tried to move back new op lower bound from 7316002925269897216 to 7316002925243092992. Current Snapshot: MvccSnapshot[applied={T|T < 7316002925269897216}]
I20260807 20:12:06.680960 23912 mvcc.cc:205] Tried to move back new op lower bound from 7316002925269897216 to 7316002925243092992. Current Snapshot: MvccSnapshot[applied={T|T < 7316002925269897216}]
I20260807 20:12:06.683007 23393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:06.683094 23393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35624:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:06.683949 23393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0c26fafbad342268913ea8e40ed9111: Sending DeleteTablet for 3 replicas of tablet bf18db2cac8b4ca88a806e513e331788
I20260807 20:12:06.684242 23526 tablet_service.cc:1514] Processing DeleteTablet for tablet bf18db2cac8b4ca88a806e513e331788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:06 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:36576
I20260807 20:12:06.684267 23670 tablet_service.cc:1514] Processing DeleteTablet for tablet bf18db2cac8b4ca88a806e513e331788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:06 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:56560
I20260807 20:12:06.684269 23811 tablet_service.cc:1514] Processing DeleteTablet for tablet bf18db2cac8b4ca88a806e513e331788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:06 UTC) from {username='kudu', principal='kudu/127.3.15.190@KRBTEST.COM'} at 127.0.0.1:41634
I20260807 20:12:06.684473 23922 tablet_replica.cc:330] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: stopping tablet replica
I20260807 20:12:06.684574 23922 raft_consensus.cc:2229] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:06.684696 23922 raft_consensus.cc:2258] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:06.684898 23920 tablet_replica.cc:330] T bf18db2cac8b4ca88a806e513e331788 P df7daaae7df24e7baa5cf73ca15f6ab7: stopping tablet replica
I20260807 20:12:06.684890 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23458
I20260807 20:12:06.686748 23921 tablet_replica.cc:330] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: stopping tablet replica
I20260807 20:12:06.686843 23921 raft_consensus.cc:2229] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:06.687136 23921 raft_consensus.cc:2258] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:06.687753 23921 ts_tablet_manager.cc:1905] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:06.687851 23922 ts_tablet_manager.cc:1905] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:06.689394 23921 ts_tablet_manager.cc:1918] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:06.689469 23921 log.cc:1196] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/bf18db2cac8b4ca88a806e513e331788
I20260807 20:12:06.689735 23921 ts_tablet_manager.cc:1939] T bf18db2cac8b4ca88a806e513e331788 P 3734ca1596d1492abe8f81c1ffc3a02b: Deleting consensus metadata
I20260807 20:12:06.690013 23922 ts_tablet_manager.cc:1918] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:06.690079 23922 log.cc:1196] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/bf18db2cac8b4ca88a806e513e331788
I20260807 20:12:06.690317 23922 ts_tablet_manager.cc:1939] T bf18db2cac8b4ca88a806e513e331788 P 8d990c478e104ee28df41c4c56fe57b4: Deleting consensus metadata
I20260807 20:12:06.690646 23380 catalog_manager.cc:4841] TS 8d990c478e104ee28df41c4c56fe57b4 (127.3.15.130:43863): tablet bf18db2cac8b4ca88a806e513e331788 (table test-table [id=8ca94f3bca1942f29cff553ca10f22da]) successfully deleted
I20260807 20:12:06.693324 23379 catalog_manager.cc:4841] TS 3734ca1596d1492abe8f81c1ffc3a02b (127.3.15.131:33089): tablet bf18db2cac8b4ca88a806e513e331788 (table test-table [id=8ca94f3bca1942f29cff553ca10f22da]) successfully deleted
W20260807 20:12:06.693759 23380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:41519 (error 108)
I20260807 20:12:06.694001 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23603
I20260807 20:12:06.694260 23893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.694149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:41519 (local address 127.0.0.1:36580)
0807 20:12:06.694194 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:06.694225 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:41519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 20:12:06.694353 23380 catalog_manager.cc:4584] TS df7daaae7df24e7baa5cf73ca15f6ab7 (127.3.15.129:41519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bf18db2cac8b4ca88a806e513e331788: Network error: Client connection negotiation failed: client connection to 127.3.15.129:41519: connect: Connection refused (error 111)
I20260807 20:12:06.699873 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23743
I20260807 20:12:06.705228 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23359
2026-08-07T20:12:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1950 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T20:12:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:06Z Disabled control of system clock
I20260807 20:12:06.735270 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34181
--webserver_interface=127.3.15.190
--webserver_port=0
--builtin_ntp_servers=127.3.15.148:40809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:34181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:06.818055 23928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:06.818272 23928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:06.818292 23928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:06.819345 23928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:06.819381 23928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:06.819393 23928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:06.819413 23928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:06.819425 23928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:06.820680 23928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:34181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:06.820940 23928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:06.821199 23928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:06.825937 23934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.825995 23936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.826028 23928 server_base.cc:1034] running on GCE node
W20260807 20:12:06.825954 23933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.826483 23928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:06.826774 23928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:06.827965 23928 hybrid_clock.cc:648] HybridClock initialized: now 1786133526827939 us; error 43 us; skew 500 ppm
I20260807 20:12:06.829747 23928 webserver.cc:460] Webserver started at http://127.3.15.190:37213/ using document root <none> and password file <none>
I20260807 20:12:06.830044 23928 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:06.830127 23928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:06.830268 23928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:06.831805 23928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "35c3c76ebb604d50ad18b65113a40095"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "efb1aaefc252ce9f1c598c919681ac2e"
server_key_iv: "0c3866979b1782d8a3cf9ce7f42a0cb4"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.832232 23928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "35c3c76ebb604d50ad18b65113a40095"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "efb1aaefc252ce9f1c598c919681ac2e"
server_key_iv: "0c3866979b1782d8a3cf9ce7f42a0cb4"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.834154 23928 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260807 20:12:06.834879 23942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.835052 23928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:06.835139 23928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "35c3c76ebb604d50ad18b65113a40095"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "efb1aaefc252ce9f1c598c919681ac2e"
server_key_iv: "0c3866979b1782d8a3cf9ce7f42a0cb4"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.835193 23928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:06.846181 23928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:06.846422 23928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:06.846519 23928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:06.850350 23928 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:34181
I20260807 20:12:06.850433 23999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:34181 every 8 connection(s)
I20260807 20:12:06.850759 23928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:06.851797 24000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:06.854166 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: Bootstrap starting.
I20260807 20:12:06.854779 24000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:06.855082 24000 log.cc:824] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:06.855773 24000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: No bootstrap required, opened a new log
I20260807 20:12:06.857542 24000 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c3c76ebb604d50ad18b65113a40095" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34181 } }
I20260807 20:12:06.857713 24000 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:06.857764 24000 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35c3c76ebb604d50ad18b65113a40095, State: Initialized, Role: FOLLOWER
I20260807 20:12:06.857928 24000 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c3c76ebb604d50ad18b65113a40095" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34181 } }
I20260807 20:12:06.858011 24000 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:06.858055 24000 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:06.858103 24000 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:06.858613 24000 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c3c76ebb604d50ad18b65113a40095" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34181 } }
I20260807 20:12:06.858731 24000 leader_election.cc:302] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35c3c76ebb604d50ad18b65113a40095; no voters:
I20260807 20:12:06.858916 24000 leader_election.cc:288] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:06.858964 24003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:06.859170 24000 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:06.859185 24003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [term 1 LEADER]: Becoming Leader. State: Replica: 35c3c76ebb604d50ad18b65113a40095, State: Running, Role: LEADER
I20260807 20:12:06.859306 24003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c3c76ebb604d50ad18b65113a40095" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34181 } }
I20260807 20:12:06.859690 24005 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35c3c76ebb604d50ad18b65113a40095. Latest consensus state: current_term: 1 leader_uuid: "35c3c76ebb604d50ad18b65113a40095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c3c76ebb604d50ad18b65113a40095" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34181 } } }
I20260807 20:12:06.859778 24004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35c3c76ebb604d50ad18b65113a40095" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35c3c76ebb604d50ad18b65113a40095" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34181 } } }
I20260807 20:12:06.859792 24005 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:06.859859 24004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:06.859880 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 23928
I20260807 20:12:06.860008 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:06.860227 3134 external_mini_cluster.cc:1428] Setting key c59b80c5e878e4b53673a6bbbcab8604
W20260807 20:12:06.861931 24023 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:06.861999 24023 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:06.862066 24010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:06.862250 24010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:06.863863 24010 catalog_manager.cc:1347] Generated new cluster ID: 34a6cc28fdf9416ca35e1e2a462e1915
I20260807 20:12:06.863919 24010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:06.866258 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:06.860935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.3.15.190:34181)
0807 20:12:06.861152 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:06.861162 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:06.861206 (+ 44us) server_negotiation.cc:407] Connection header received
0807 20:12:06.861343 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:06.861350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:06.861432 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:06.861845 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:06.861948 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.863361 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.864268 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:06.864700 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:06.864870 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:06.865563 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:06.865567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:06.865567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:06.865762 (+ 195us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:06.865796 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:06.865807 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:06.865885 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:06.865961 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 20:12:06.866045 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
I20260807 20:12:06.880600 24010 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:06.881316 24010 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:06.889389 24010 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: Generated new TSK 0
I20260807 20:12:06.889559 24010 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:06.890882 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.129:0
--local_ip_for_outbound_sockets=127.3.15.129
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34181
--builtin_ntp_servers=127.3.15.148:40809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 20:12:06.975900 24026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:06.976157 24026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:06.976197 24026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:06.977460 24026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:06.977522 24026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:06.977566 24026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.129
I20260807 20:12:06.978976 24026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.15.129
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.129
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:06.979259 24026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:06.979573 24026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:06.985651 24035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.985663 24033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:06.985651 24032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:06.986496 24026 server_base.cc:1034] running on GCE node
I20260807 20:12:06.986752 24026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:06.986996 24026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:06.988154 24026 hybrid_clock.cc:648] HybridClock initialized: now 1786133526988141 us; error 32 us; skew 500 ppm
I20260807 20:12:06.989461 24026 webserver.cc:460] Webserver started at http://127.3.15.129:33975/ using document root <none> and password file <none>
I20260807 20:12:06.989748 24026 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:06.989820 24026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:06.989977 24026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:06.991053 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data/instance:
uuid: "653e29e39ac74bc8a850e826e63c13d1"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "633a432b51541f5c8310bdfb41d31929"
server_key_iv: "d0c7943853c30273be57f1db3f89a403"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.991425 24026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance:
uuid: "653e29e39ac74bc8a850e826e63c13d1"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "633a432b51541f5c8310bdfb41d31929"
server_key_iv: "d0c7943853c30273be57f1db3f89a403"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.992837 24026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:06.993671 24041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:06.993852 24026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:06.993954 24026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal
uuid: "653e29e39ac74bc8a850e826e63c13d1"
format_stamp: "Formatted at 2026-08-07 20:12:06 on dist-test-slave-1x4g"
server_key: "633a432b51541f5c8310bdfb41d31929"
server_key_iv: "d0c7943853c30273be57f1db3f89a403"
server_key_version: "encryptionkey@0"
I20260807 20:12:06.994042 24026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:07.014916 24026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:07.015242 24026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:07.015403 24026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:07.015645 24026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:07.015983 24026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:07.016037 24026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.016072 24026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:07.016099 24026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.022058 24026 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.129:44145
I20260807 20:12:07.022147 24166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.129:44145 every 8 connection(s)
I20260807 20:12:07.022495 24026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/data/info.pb
I20260807 20:12:07.027531 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24026
I20260807 20:12:07.027637 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-0/wal/instance
I20260807 20:12:07.027661 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.023255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.129:47725 (local address 127.3.15.190:34181)
0807 20:12:07.023300 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.023305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.023561 (+ 256us) server_negotiation.cc:407] Connection header received
0807 20:12:07.024053 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.024054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.024063 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.024125 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:07.024837 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.025344 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.026293 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.026476 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.026529 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.027383 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:07.027384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:07.027384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:07.027389 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:07.027401 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:07.027404 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:07.027445 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.027532 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.027554 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:12:07.027841 3134 external_mini_cluster.cc:1428] Setting key 491069017b7e3576a93a97d16bf93303
I20260807 20:12:07.027891 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.023231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34181 (local address 127.3.15.129:47725)
0807 20:12:07.023428 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.023470 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.023971 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.024181 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.024185 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.024194 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:07.024719 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.024723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.025428 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.025429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.026221 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.026224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.026554 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.026554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.026675 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.027237 (+ 562us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:07.027237 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:07.027266 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:07.027289 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:07.027448 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:07.027454 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:07.027457 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.027530 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.027657 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260807 20:12:07.028427 24167 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34181
I20260807 20:12:07.028537 24167 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:07.028743 24167 heartbeater.cc:511] Master 127.3.15.190:34181 requested a full tablet report, sending...
I20260807 20:12:07.028862 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.130:0
--local_ip_for_outbound_sockets=127.3.15.130
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34181
--builtin_ntp_servers=127.3.15.148:40809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:12:07.029387 23963 ts_manager.cc:194] Registered new tserver with Master: 653e29e39ac74bc8a850e826e63c13d1 (127.3.15.129:44145)
I20260807 20:12:07.030296 23963 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.129:47725
W20260807 20:12:07.113447 24171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:07.113682 24171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:07.113706 24171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:07.114923 24171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:07.114964 24171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:07.114991 24171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.130
I20260807 20:12:07.116307 24171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.15.130
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24171
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.130
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:07.116506 24171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:07.116861 24171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:07.121965 24176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:07.122090 24171 server_base.cc:1034] running on GCE node
W20260807 20:12:07.121990 24177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:07.122193 24179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:07.122411 24171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:07.122623 24171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:07.123764 24171 hybrid_clock.cc:648] HybridClock initialized: now 1786133527123735 us; error 54 us; skew 500 ppm
I20260807 20:12:07.125093 24171 webserver.cc:460] Webserver started at http://127.3.15.130:37607/ using document root <none> and password file <none>
I20260807 20:12:07.125339 24171 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:07.125387 24171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:07.125548 24171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:07.126653 24171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data/instance:
uuid: "f9276d8451b745739610c10a7eeeda68"
format_stamp: "Formatted at 2026-08-07 20:12:07 on dist-test-slave-1x4g"
server_key: "a20922866fa3912d4602234627c72072"
server_key_iv: "ad1f0b8db56c1d6b83fd4c1a90416cab"
server_key_version: "encryptionkey@0"
I20260807 20:12:07.127023 24171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance:
uuid: "f9276d8451b745739610c10a7eeeda68"
format_stamp: "Formatted at 2026-08-07 20:12:07 on dist-test-slave-1x4g"
server_key: "a20922866fa3912d4602234627c72072"
server_key_iv: "ad1f0b8db56c1d6b83fd4c1a90416cab"
server_key_version: "encryptionkey@0"
I20260807 20:12:07.128345 24171 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:07.129132 24186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.129307 24171 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.129400 24171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal
uuid: "f9276d8451b745739610c10a7eeeda68"
format_stamp: "Formatted at 2026-08-07 20:12:07 on dist-test-slave-1x4g"
server_key: "a20922866fa3912d4602234627c72072"
server_key_iv: "ad1f0b8db56c1d6b83fd4c1a90416cab"
server_key_version: "encryptionkey@0"
I20260807 20:12:07.129475 24171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:07.137352 24171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:07.137655 24171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:07.137807 24171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:07.138029 24171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:07.138343 24171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:07.138396 24171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.138435 24171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:07.138466 24171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.144256 24171 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.130:41739
I20260807 20:12:07.144304 24309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.130:41739 every 8 connection(s)
I20260807 20:12:07.144621 24171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/data/info.pb
I20260807 20:12:07.144950 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24171
I20260807 20:12:07.145058 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal/instance
I20260807 20:12:07.145246 3134 external_mini_cluster.cc:1428] Setting key 882308ac4589bb076c28096c0ded0a58
I20260807 20:12:07.146214 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.15.131:0
--local_ip_for_outbound_sockets=127.3.15.131
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34181
--builtin_ntp_servers=127.3.15.148:40809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 20:12:07.149976 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.145375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.130:50225 (local address 127.3.15.190:34181)
0807 20:12:07.145445 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.145449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.145638 (+ 189us) server_negotiation.cc:407] Connection header received
0807 20:12:07.146200 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.146201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.146217 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.146273 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:07.146957 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.147457 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.148428 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.148639 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.148716 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.149720 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:07.149722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:07.149722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:07.149726 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:07.149740 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:07.149745 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:07.149792 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.149864 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.149896 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 20:12:07.150194 24314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.145344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34181 (local address 127.3.15.130:50225)
0807 20:12:07.145522 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.145568 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.146091 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.146351 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.146354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.146362 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:07.146857 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.146861 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.147555 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.147556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.148337 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.148339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.148726 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.148727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.148909 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.149525 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:07.149526 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:07.149553 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:07.149617 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:07.149792 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:07.149798 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:07.149801 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.149867 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.149968 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260807 20:12:07.150779 24310 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34181
I20260807 20:12:07.150890 24310 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:07.151105 24310 heartbeater.cc:511] Master 127.3.15.190:34181 requested a full tablet report, sending...
I20260807 20:12:07.151511 23963 ts_manager.cc:194] Registered new tserver with Master: f9276d8451b745739610c10a7eeeda68 (127.3.15.130:41739)
I20260807 20:12:07.151880 23963 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.130:50225
W20260807 20:12:07.231029 24315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:07.231238 24315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:07.231259 24315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:07.232551 24315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:07.232599 24315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:07.232622 24315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.15.131
I20260807 20:12:07.234135 24315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:40809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.15.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.15.131
--webserver_port=0
--tserver_master_addrs=127.3.15.190:34181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.15.131
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:07.234390 24315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:07.234699 24315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:07.239727 24323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:07.239727 24324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:07.239738 24326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:07.240001 24315 server_base.cc:1034] running on GCE node
I20260807 20:12:07.240257 24315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:07.240523 24315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:07.241760 24315 hybrid_clock.cc:648] HybridClock initialized: now 1786133527241746 us; error 29 us; skew 500 ppm
I20260807 20:12:07.243170 24315 webserver.cc:460] Webserver started at http://127.3.15.131:40347/ using document root <none> and password file <none>
I20260807 20:12:07.243420 24315 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:07.243467 24315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:07.243630 24315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:07.244733 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data/instance:
uuid: "c1bebc43c1f042e08e15180f618f8675"
format_stamp: "Formatted at 2026-08-07 20:12:07 on dist-test-slave-1x4g"
server_key: "d4d20fd2f077055926f6082d42326345"
server_key_iv: "06822f43c8ffb425b0c7b542ef79ec20"
server_key_version: "encryptionkey@0"
I20260807 20:12:07.245175 24315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance:
uuid: "c1bebc43c1f042e08e15180f618f8675"
format_stamp: "Formatted at 2026-08-07 20:12:07 on dist-test-slave-1x4g"
server_key: "d4d20fd2f077055926f6082d42326345"
server_key_iv: "06822f43c8ffb425b0c7b542ef79ec20"
server_key_version: "encryptionkey@0"
I20260807 20:12:07.246622 24315 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:07.247404 24332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.247577 24315 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:07.247692 24315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal
uuid: "c1bebc43c1f042e08e15180f618f8675"
format_stamp: "Formatted at 2026-08-07 20:12:07 on dist-test-slave-1x4g"
server_key: "d4d20fd2f077055926f6082d42326345"
server_key_iv: "06822f43c8ffb425b0c7b542ef79ec20"
server_key_version: "encryptionkey@0"
I20260807 20:12:07.247774 24315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:07.260152 24315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:07.260433 24315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:07.260576 24315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:07.260857 24315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 20:12:07.261240 24315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 20:12:07.261296 24315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.261336 24315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 20:12:07.261364 24315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:07.267001 24315 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.131:32943
I20260807 20:12:07.267076 24453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.131:32943 every 8 connection(s)
I20260807 20:12:07.267414 24315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/data/info.pb
I20260807 20:12:07.272019 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24315
I20260807 20:12:07.272146 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal/instance
I20260807 20:12:07.272425 3134 external_mini_cluster.cc:1428] Setting key fef825f8da5d2f730cdc22076818496f
I20260807 20:12:07.273010 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.268153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:32857 (local address 127.3.15.190:34181)
0807 20:12:07.268220 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.268222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.268456 (+ 234us) server_negotiation.cc:407] Connection header received
0807 20:12:07.269068 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.269070 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.269084 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.269144 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:07.269828 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.270343 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.271426 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.271641 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.271728 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.272653 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:07.272655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:07.272655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:07.272675 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:07.272695 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:07.272700 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:07.272790 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.272902 (+ 112us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.272932 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:12:07.273298 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.268146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:34181 (local address 127.3.15.131:32857)
0807 20:12:07.268343 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.268376 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.268948 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.269219 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.269222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.269230 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:07.269730 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.269734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.270441 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.270442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.271329 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.271332 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.271739 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.271740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.271868 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.272475 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 20:12:07.272476 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:07.272518 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 20:12:07.272548 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:07.272806 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:07.272814 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:07.272817 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.272890 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.273003 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":89,"threads_started":1}
I20260807 20:12:07.273720 24455 heartbeater.cc:347] Connected to a master server at 127.3.15.190:34181
I20260807 20:12:07.273811 24455 heartbeater.cc:464] Registering TS with master...
I20260807 20:12:07.273957 24455 heartbeater.cc:511] Master 127.3.15.190:34181 requested a full tablet report, sending...
I20260807 20:12:07.274273 23963 ts_manager.cc:194] Registered new tserver with Master: c1bebc43c1f042e08e15180f618f8675 (127.3.15.131:32943)
I20260807 20:12:07.274600 23963 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.15.131:32857
I20260807 20:12:07.284441 3134 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 20:12:07.288379 24017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.285850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.3.15.190:34181)
0807 20:12:07.285908 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.285910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.286019 (+ 109us) server_negotiation.cc:407] Connection header received
0807 20:12:07.286064 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.286065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.286074 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.286133 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:07.286261 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.286707 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.287406 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.287570 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.287618 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.288159 (+ 541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:07.288160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:07.288161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:07.288165 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:07.288177 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:07.288181 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:07.288218 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.288257 (+ 39us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.288275 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 20:12:07.289868 23963 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 20:12:07.291317 23963 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 20:12:07.297492 24464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.293297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:41739 (local address 127.0.0.1:60210)
0807 20:12:07.293436 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.293479 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.293544 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.293892 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.293895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.293897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:07.293961 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.293963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.295290 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.295291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.296338 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.296340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.297195 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.297196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.297277 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.297291 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.297358 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.297416 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":90,"threads_started":1}
I20260807 20:12:07.297590 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.293341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:44145 (local address 127.0.0.1:50916)
0807 20:12:07.293702 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.293714 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.293772 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.294122 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.294123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.294125 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:07.294171 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.294173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.295359 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.295360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.296225 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.296234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.297334 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.297335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.297401 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.297404 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.297458 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.297508 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":4,"thread_start_us":97,"threads_started":1}
I20260807 20:12:07.297699 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.293710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.3.15.131:32943)
0807 20:12:07.293883 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.293887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.293896 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:07.293929 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.293930 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.293946 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.294016 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:07.294252 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.294933 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.296391 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.297180 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.297238 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.297323 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.297551 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.297590 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260807 20:12:07.297817 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.293479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.3.15.129:44145)
0807 20:12:07.293893 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.293898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.293907 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:07.293947 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.293949 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.293964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.294040 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:07.294256 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.295300 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.296315 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.297123 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.297198 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.297293 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.297689 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.297735 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":101,"threads_started":1}
I20260807 20:12:07.297833 24467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.293528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.131:32943 (local address 127.0.0.1:57244)
0807 20:12:07.293723 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.293734 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.293772 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.294119 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.294120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.294122 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:07.294167 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.294169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.295025 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.295026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.296312 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.296316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.297352 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.297353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.297401 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.297407 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.297458 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.297508 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":7,"thread_start_us":65,"threads_started":1}
I20260807 20:12:07.298130 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.293511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60210 (local address 127.3.15.130:41739)
0807 20:12:07.293719 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.293724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.293735 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:12:07.293779 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.293781 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.293798 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.293886 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:07.294061 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.295203 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.296446 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.297070 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.297166 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.297264 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.297394 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.297434 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":86,"threads_started":1}
I20260807 20:12:07.298604 24091 tablet_service.cc:1467] Processing CreateTablet for tablet 2667f50ae7094c61a1d4d9c5fcd96759 (DEFAULT_TABLE table=test-table [id=4b9365b29a534a4da01a5e03fd1e832f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:07.298722 24380 tablet_service.cc:1467] Processing CreateTablet for tablet 2667f50ae7094c61a1d4d9c5fcd96759 (DEFAULT_TABLE table=test-table [id=4b9365b29a534a4da01a5e03fd1e832f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:07.298730 24236 tablet_service.cc:1467] Processing CreateTablet for tablet 2667f50ae7094c61a1d4d9c5fcd96759 (DEFAULT_TABLE table=test-table [id=4b9365b29a534a4da01a5e03fd1e832f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 20:12:07.299012 24091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2667f50ae7094c61a1d4d9c5fcd96759. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:07.299014 24380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2667f50ae7094c61a1d4d9c5fcd96759. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:07.299084 24236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2667f50ae7094c61a1d4d9c5fcd96759. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:07.301784 24470 tablet_bootstrap.cc:492] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Bootstrap starting.
I20260807 20:12:07.302356 24472 tablet_bootstrap.cc:492] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: Bootstrap starting.
I20260807 20:12:07.302706 24470 tablet_bootstrap.cc:654] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:07.303014 24472 tablet_bootstrap.cc:654] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:07.303042 24470 log.cc:824] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:07.303351 24472 log.cc:824] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:07.303689 24471 tablet_bootstrap.cc:492] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Bootstrap starting.
I20260807 20:12:07.303877 24470 tablet_bootstrap.cc:492] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: No bootstrap required, opened a new log
I20260807 20:12:07.303958 24470 ts_tablet_manager.cc:1397] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:07.304234 24472 tablet_bootstrap.cc:492] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: No bootstrap required, opened a new log
I20260807 20:12:07.304317 24472 ts_tablet_manager.cc:1397] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:07.304348 24471 tablet_bootstrap.cc:654] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:07.304672 24471 log.cc:824] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:07.305531 24471 tablet_bootstrap.cc:492] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: No bootstrap required, opened a new log
I20260807 20:12:07.305604 24471 ts_tablet_manager.cc:1397] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 20:12:07.306346 24472 raft_consensus.cc:348] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.306504 24472 raft_consensus.cc:374] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:07.306547 24472 raft_consensus.cc:729] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653e29e39ac74bc8a850e826e63c13d1, State: Initialized, Role: FOLLOWER
I20260807 20:12:07.306675 24472 consensus_queue.cc:260] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.306735 24470 raft_consensus.cc:348] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.306911 24470 raft_consensus.cc:374] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:07.306957 24470 raft_consensus.cc:729] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9276d8451b745739610c10a7eeeda68, State: Initialized, Role: FOLLOWER
I20260807 20:12:07.306957 24472 ts_tablet_manager.cc:1428] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 20:12:07.307088 24167 heartbeater.cc:503] Master 127.3.15.190:34181 was elected leader, sending a full tablet report...
I20260807 20:12:07.307091 24470 consensus_queue.cc:260] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.307335 24470 ts_tablet_manager.cc:1428] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 20:12:07.307418 24310 heartbeater.cc:503] Master 127.3.15.190:34181 was elected leader, sending a full tablet report...
I20260807 20:12:07.308641 24471 raft_consensus.cc:348] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.308759 24471 raft_consensus.cc:374] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:07.308845 24471 raft_consensus.cc:729] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1bebc43c1f042e08e15180f618f8675, State: Initialized, Role: FOLLOWER
I20260807 20:12:07.308965 24471 consensus_queue.cc:260] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.309283 24471 ts_tablet_manager.cc:1428] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 20:12:07.309309 24455 heartbeater.cc:503] Master 127.3.15.190:34181 was elected leader, sending a full tablet report...
W20260807 20:12:07.395223 24313 tablet.cc:2369] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:07.478901 24480 raft_consensus.cc:482] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 20:12:07.479030 24480 raft_consensus.cc:504] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.479349 24480 leader_election.cc:288] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9276d8451b745739610c10a7eeeda68 (127.3.15.130:41739), 653e29e39ac74bc8a850e826e63c13d1 (127.3.15.129:44145)
I20260807 20:12:07.483269 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.479628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:54155 (local address 127.3.15.130:41739)
0807 20:12:07.479671 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.479673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.479683 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:07.479750 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.479750 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.479777 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.479838 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:07.480032 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.480828 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.482337 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.482896 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.482975 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.482984 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.483157 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.483198 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:12:07.483269 24466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.479619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.15.131:54877 (local address 127.3.15.129:44145)
0807 20:12:07.479669 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.479672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.479785 (+ 113us) server_negotiation.cc:407] Connection header received
0807 20:12:07.479816 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.479817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.479829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.479887 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 20:12:07.480035 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.480834 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.482337 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.482896 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.482975 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.482984 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.483157 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.483198 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 20:12:07.483266 24457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.479531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.130:41739 (local address 127.3.15.131:54155)
0807 20:12:07.479601 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.479616 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.479659 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.479858 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.479859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.479860 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:07.479946 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.479948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.480916 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.480917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.482238 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.482242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.482986 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.482987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.483063 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.483068 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.483128 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.483168 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":44}
I20260807 20:12:07.483279 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.479544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:44145 (local address 127.3.15.131:54877)
0807 20:12:07.479714 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.479723 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.479778 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.479885 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:07.479886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:07.479887 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 20:12:07.479946 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.479948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.480916 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.480917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.482238 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:07.482242 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.482986 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.482987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:07.483068 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.483070 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 20:12:07.483134 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 20:12:07.483166 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":65,"thread_start_us":101,"threads_started":1}
I20260807 20:12:07.483688 24117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2667f50ae7094c61a1d4d9c5fcd96759" candidate_uuid: "c1bebc43c1f042e08e15180f618f8675" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "653e29e39ac74bc8a850e826e63c13d1" is_pre_election: true
I20260807 20:12:07.483700 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2667f50ae7094c61a1d4d9c5fcd96759" candidate_uuid: "c1bebc43c1f042e08e15180f618f8675" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9276d8451b745739610c10a7eeeda68" is_pre_election: true
I20260807 20:12:07.483858 24117 raft_consensus.cc:2454] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1bebc43c1f042e08e15180f618f8675 in term 0.
I20260807 20:12:07.483858 24260 raft_consensus.cc:2454] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1bebc43c1f042e08e15180f618f8675 in term 0.
I20260807 20:12:07.484102 24336 leader_election.cc:302] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c1bebc43c1f042e08e15180f618f8675, f9276d8451b745739610c10a7eeeda68; no voters:
I20260807 20:12:07.484274 24480 raft_consensus.cc:2781] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 20:12:07.484361 24480 raft_consensus.cc:482] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 20:12:07.484396 24480 raft_consensus.cc:3037] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:07.485230 24480 raft_consensus.cc:504] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.485387 24480 leader_election.cc:288] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [CANDIDATE]: Term 1 election: Requested vote from peers f9276d8451b745739610c10a7eeeda68 (127.3.15.130:41739), 653e29e39ac74bc8a850e826e63c13d1 (127.3.15.129:44145)
I20260807 20:12:07.485528 24117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2667f50ae7094c61a1d4d9c5fcd96759" candidate_uuid: "c1bebc43c1f042e08e15180f618f8675" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "653e29e39ac74bc8a850e826e63c13d1"
I20260807 20:12:07.485561 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2667f50ae7094c61a1d4d9c5fcd96759" candidate_uuid: "c1bebc43c1f042e08e15180f618f8675" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9276d8451b745739610c10a7eeeda68"
I20260807 20:12:07.485610 24117 raft_consensus.cc:3037] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:07.485632 24260 raft_consensus.cc:3037] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:07.486439 24117 raft_consensus.cc:2454] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1bebc43c1f042e08e15180f618f8675 in term 1.
I20260807 20:12:07.486492 24260 raft_consensus.cc:2454] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1bebc43c1f042e08e15180f618f8675 in term 1.
I20260807 20:12:07.486683 24333 leader_election.cc:302] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 653e29e39ac74bc8a850e826e63c13d1, c1bebc43c1f042e08e15180f618f8675; no voters:
I20260807 20:12:07.486801 24480 raft_consensus.cc:2781] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:07.486949 24480 raft_consensus.cc:686] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 1 LEADER]: Becoming Leader. State: Replica: c1bebc43c1f042e08e15180f618f8675, State: Running, Role: LEADER
I20260807 20:12:07.487051 24480 consensus_queue.cc:237] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } }
I20260807 20:12:07.487938 23961 catalog_manager.cc:5495] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 reported cstate change: term changed from 0 to 1, leader changed from <none> to c1bebc43c1f042e08e15180f618f8675 (127.3.15.131). New cstate: current_term: 1 leader_uuid: "c1bebc43c1f042e08e15180f618f8675" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1bebc43c1f042e08e15180f618f8675" member_type: VOTER last_known_addr { host: "127.3.15.131" port: 32943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 } health_report { overall_health: UNKNOWN } } }
W20260807 20:12:07.517987 24456 tablet.cc:2369] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 20:12:07.523232 24168 tablet.cc:2369] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 20:12:07.526000 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:07.523658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57248 (local address 127.3.15.131:32943)
0807 20:12:07.523731 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:07.523733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:07.523741 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:07.523779 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:07.523780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:07.523799 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:07.523861 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:07.523991 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.524452 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.525260 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:07.525445 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:07.525504 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:07.525653 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:07.525756 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:07.525817 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:07.525848 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 20:12:07.525865 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 20:12:07.527905 24117 raft_consensus.cc:1261] T 2667f50ae7094c61a1d4d9c5fcd96759 P 653e29e39ac74bc8a850e826e63c13d1 [term 1 FOLLOWER]: Refusing update from remote peer c1bebc43c1f042e08e15180f618f8675: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:07.527959 24260 raft_consensus.cc:1261] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 1 FOLLOWER]: Refusing update from remote peer c1bebc43c1f042e08e15180f618f8675: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:07.528329 24480 consensus_queue.cc:1035] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9276d8451b745739610c10a7eeeda68" member_type: VOTER last_known_addr { host: "127.3.15.130" port: 41739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:07.528476 24482 consensus_queue.cc:1035] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [LEADER]: Connected to new peer: Peer: permanent_uuid: "653e29e39ac74bc8a850e826e63c13d1" member_type: VOTER last_known_addr { host: "127.3.15.129" port: 44145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:07.530915 24485 mvcc.cc:205] Tried to move back new op lower bound from 7316002928751472640 to 7316002928587669504. Current Snapshot: MvccSnapshot[applied={T|T < 7316002928751472640}]
I20260807 20:12:07.531395 24487 mvcc.cc:205] Tried to move back new op lower bound from 7316002928751472640 to 7316002928587669504. Current Snapshot: MvccSnapshot[applied={T|T < 7316002928751472640}]
I20260807 20:12:07.533598 23961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:07.533704 23961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 20:12:07.534456 23961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35c3c76ebb604d50ad18b65113a40095: Sending DeleteTablet for 3 replicas of tablet 2667f50ae7094c61a1d4d9c5fcd96759
I20260807 20:12:07.534802 24380 tablet_service.cc:1514] Processing DeleteTablet for tablet 2667f50ae7094c61a1d4d9c5fcd96759 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:07 UTC) from {username='slave'} at 127.0.0.1:57244
I20260807 20:12:07.534781 24091 tablet_service.cc:1514] Processing DeleteTablet for tablet 2667f50ae7094c61a1d4d9c5fcd96759 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:07 UTC) from {username='slave'} at 127.0.0.1:50916
I20260807 20:12:07.535036 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24026
I20260807 20:12:07.535072 24236 tablet_service.cc:1514] Processing DeleteTablet for tablet 2667f50ae7094c61a1d4d9c5fcd96759 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 20:12:07 UTC) from {username='slave'} at 127.0.0.1:60210
I20260807 20:12:07.535660 24495 tablet_replica.cc:330] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: stopping tablet replica
I20260807 20:12:07.535746 24495 raft_consensus.cc:2229] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 1 LEADER]: Raft consensus shutting down.
I20260807 20:12:07.535825 24495 raft_consensus.cc:2258] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:07.536510 24495 ts_tablet_manager.cc:1905] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:07.538645 24495 ts_tablet_manager.cc:1918] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:07.538712 24495 log.cc:1196] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-2/wal/wals/2667f50ae7094c61a1d4d9c5fcd96759
I20260807 20:12:07.539011 24495 ts_tablet_manager.cc:1939] T 2667f50ae7094c61a1d4d9c5fcd96759 P c1bebc43c1f042e08e15180f618f8675: Deleting consensus metadata
I20260807 20:12:07.539438 23946 catalog_manager.cc:4841] TS c1bebc43c1f042e08e15180f618f8675 (127.3.15.131:32943): tablet 2667f50ae7094c61a1d4d9c5fcd96759 (table test-table [id=4b9365b29a534a4da01a5e03fd1e832f]) successfully deleted
I20260807 20:12:07.539892 24497 tablet_replica.cc:330] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: stopping tablet replica
I20260807 20:12:07.540057 24497 raft_consensus.cc:2229] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 20:12:07.540227 24497 raft_consensus.cc:2258] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 20:12:07.540717 24497 ts_tablet_manager.cc:1905] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 20:12:07.542121 24497 ts_tablet_manager.cc:1918] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 20:12:07.542210 24497 log.cc:1196] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Deleting WAL directory at /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786133281907902-3134-0/minicluster-data/ts-1/wal/wals/2667f50ae7094c61a1d4d9c5fcd96759
I20260807 20:12:07.542461 24497 ts_tablet_manager.cc:1939] T 2667f50ae7094c61a1d4d9c5fcd96759 P f9276d8451b745739610c10a7eeeda68: Deleting consensus metadata
I20260807 20:12:07.542865 23946 catalog_manager.cc:4841] TS f9276d8451b745739610c10a7eeeda68 (127.3.15.130:41739): tablet 2667f50ae7094c61a1d4d9c5fcd96759 (table test-table [id=4b9365b29a534a4da01a5e03fd1e832f]) successfully deleted
W20260807 20:12:07.545847 23943 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.15.129:44145 (error 108)
W20260807 20:12:07.546396 24467 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:07.546203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.129:44145 (local address 127.0.0.1:50924)
0807 20:12:07.546263 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:07.546274 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:07.546323 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:07.546360 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.129:44145: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 20:12:07.546535 23943 catalog_manager.cc:4584] TS 653e29e39ac74bc8a850e826e63c13d1 (127.3.15.129:44145): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2667f50ae7094c61a1d4d9c5fcd96759: Network error: Client connection negotiation failed: client connection to 127.3.15.129:44145: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260807 20:12:07.546664 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24171
I20260807 20:12:07.554536 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24315
I20260807 20:12:07.559938 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 23928
2026-08-07T20:12:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (853 ms)
[----------] 6 tests from EncryptionPolicyTest (8732 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:07 dist-test-slave-1x4g krb5kdc[24501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:07 dist-test-slave-1x4g krb5kdc[24501](info): set up 2 sockets
Aug 07 20:12:07 dist-test-slave-1x4g krb5kdc[24501](info): commencing operation
krb5kdc: starting...
W20260807 20:12:08.645493 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:08 dist-test-slave-1x4g krb5kdc[24501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:08.767668 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:08.859146 24515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:08.859350 24515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:08.859373 24515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:08.860530 24515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:08.860574 24515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:08.860589 24515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:08.860646 24515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:08.860663 24515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:08.862193 24515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:08.862453 24515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:08.862711 24515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:08.867549 24524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:08.867662 24520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:08.867559 24522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:08.867890 24515 server_base.cc:1034] running on GCE node
I20260807 20:12:08.868125 24515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:08.868412 24515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:08.869597 24515 hybrid_clock.cc:648] HybridClock initialized: now 1786133528869571 us; error 32 us; skew 500 ppm
Aug 07 20:12:08 dist-test-slave-1x4g krb5kdc[24501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133528, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:08.872022 24515 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:08.872541 24515 webserver.cc:460] Webserver started at http://127.0.0.1:37785/ using document root <none> and password file <none>
I20260807 20:12:08.872825 24515 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:08.872890 24515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:08.873029 24515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:08.874112 24515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "f2062f32f6664986a617b99a09276403"
format_stamp: "Formatted at 2026-08-07 20:12:08 on dist-test-slave-1x4g"
server_key: "433e3625cbba0d1ed52cfd35ed2185f1"
server_key_iv: "eb1385d309994daa7a6cd031096e9bd9"
server_key_version: "encryptionkey@0"
I20260807 20:12:08.874496 24515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "f2062f32f6664986a617b99a09276403"
format_stamp: "Formatted at 2026-08-07 20:12:08 on dist-test-slave-1x4g"
server_key: "433e3625cbba0d1ed52cfd35ed2185f1"
server_key_iv: "eb1385d309994daa7a6cd031096e9bd9"
server_key_version: "encryptionkey@0"
I20260807 20:12:08.875829 24515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:08.876587 24531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:08.876837 24515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:08.876942 24515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "f2062f32f6664986a617b99a09276403"
format_stamp: "Formatted at 2026-08-07 20:12:08 on dist-test-slave-1x4g"
server_key: "433e3625cbba0d1ed52cfd35ed2185f1"
server_key_iv: "eb1385d309994daa7a6cd031096e9bd9"
server_key_version: "encryptionkey@0"
I20260807 20:12:08.877028 24515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:08.901185 24515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:08.903151 24515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:08.903321 24515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:08.907697 24515 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42743
I20260807 20:12:08.907758 24585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42743 every 8 connection(s)
I20260807 20:12:08.908149 24515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:08.909169 24586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:08.911464 24586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403: Bootstrap starting.
I20260807 20:12:08.912009 24586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:08.912259 24586 log.cc:824] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:08.912969 24586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403: No bootstrap required, opened a new log
I20260807 20:12:08.913374 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24515
I20260807 20:12:08.913494 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:08.913717 3134 external_mini_cluster.cc:1428] Setting key 69141c0fe1902734ff06d71fc70bafdb
I20260807 20:12:08.915175 24586 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2062f32f6664986a617b99a09276403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260807 20:12:08.915352 24586 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:08.915393 24586 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2062f32f6664986a617b99a09276403, State: Initialized, Role: FOLLOWER
I20260807 20:12:08.915575 24586 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2062f32f6664986a617b99a09276403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260807 20:12:08.915676 24586 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:08.915712 24586 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:08.915767 24586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:08.916901 24586 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2062f32f6664986a617b99a09276403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260807 20:12:08.916996 24586 leader_election.cc:302] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2062f32f6664986a617b99a09276403; no voters:
I20260807 20:12:08.917187 24586 leader_election.cc:288] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:08.917209 24591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:08.917323 24591 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [term 1 LEADER]: Becoming Leader. State: Replica: f2062f32f6664986a617b99a09276403, State: Running, Role: LEADER
I20260807 20:12:08.917466 24586 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:08.917498 24591 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2062f32f6664986a617b99a09276403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } }
I20260807 20:12:08.917915 24592 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2062f32f6664986a617b99a09276403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2062f32f6664986a617b99a09276403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } } }
I20260807 20:12:08.918011 24592 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:08.917927 24593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2062f32f6664986a617b99a09276403. Latest consensus state: current_term: 1 leader_uuid: "f2062f32f6664986a617b99a09276403" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2062f32f6664986a617b99a09276403" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42743 } } }
I20260807 20:12:08.918056 24593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:08.919081 24597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:08.919816 24597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:08.921325 24597 catalog_manager.cc:1347] Generated new cluster ID: 727be7a7285d4e1db647c38e9d84ebce
I20260807 20:12:08.921382 24597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:08 dist-test-slave-1x4g krb5kdc[24501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:12:08.925740 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:08.914270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36291 (local address 127.0.0.1:42743)
0807 20:12:08.914462 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:08.914470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:08.914495 (+ 25us) server_negotiation.cc:407] Connection header received
0807 20:12:08.915157 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:08.915164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:08.915210 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:08.915325 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:08.916181 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:08.917344 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:08.918394 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:08.919017 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:08.919507 (+ 490us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:08.922145 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:08.922149 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:08.922149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:08.922163 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:08.923851 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:08.924258 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:08.924264 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:08.924264 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:08.924380 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:08.924629 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:08.924630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:08.924631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:08.924977 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:08.925158 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:08.925285 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 20:12:08.925395 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260807 20:12:08.926783 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:08.931612 24597 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:08.932302 24597 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:08.935443 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:08.928162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58469 (local address 127.0.0.1:42743)
0807 20:12:08.928202 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:08.928204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:08.928942 (+ 738us) server_negotiation.cc:407] Connection header received
0807 20:12:08.929016 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:08.929017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:08.929027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:08.929079 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:08.929590 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:08.930401 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:08.931154 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:08.931474 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:08.931555 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:08.932841 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:08.932842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:08.932843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:08.932850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:08.934250 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:08.934622 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:08.934623 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:08.934624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:08.934684 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:08.934914 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:08.934915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:08.934915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:08.935027 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:08.935141 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:08.935269 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:12:08.935332 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":306,"server-negotiator.queue_time_us":19}
I20260807 20:12:08.940862 24597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2062f32f6664986a617b99a09276403: Generated new TSK 0
I20260807 20:12:08.941105 24597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:08.943044 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24515
W20260807 20:12:08.951936 24499 connection.cc:535] client connection to 127.0.0.1:42743 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42743): Connection reset by peer (error 104)
2026-08-07T20:12:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:09 dist-test-slave-1x4g krb5kdc[24621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:09 dist-test-slave-1x4g krb5kdc[24621](info): set up 2 sockets
Aug 07 20:12:09 dist-test-slave-1x4g krb5kdc[24621](info): commencing operation
krb5kdc: starting...
W20260807 20:12:10.013249 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:10 dist-test-slave-1x4g krb5kdc[24621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:10.139122 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34697
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:10.223536 24635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:10.223789 24635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:10.223830 24635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:10.225011 24635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:10.225086 24635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:10.225109 24635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:10.225129 24635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:10.225160 24635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:10.226665 24635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34697
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:10.226969 24635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:10.227203 24635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:10.232218 24644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:10.232414 24635 server_base.cc:1034] running on GCE node
W20260807 20:12:10.232218 24641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:10.232220 24642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:10.233036 24635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:10.233407 24635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:10.234623 24635 hybrid_clock.cc:648] HybridClock initialized: now 1786133530234598 us; error 43 us; skew 500 ppm
Aug 07 20:12:10 dist-test-slave-1x4g krb5kdc[24621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133530, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:10.237277 24635 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:10.237792 24635 webserver.cc:460] Webserver started at http://127.0.0.1:46521/ using document root <none> and password file <none>
I20260807 20:12:10.238052 24635 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:10.238134 24635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:10.238298 24635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:10.239416 24635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "2fe67533e43d4624bb16596401a1fb95"
format_stamp: "Formatted at 2026-08-07 20:12:10 on dist-test-slave-1x4g"
server_key: "6dac2d6df6fb8de2858c681158716396"
server_key_iv: "aadd2bf9d85ad5dbec9c27ec906e4564"
server_key_version: "encryptionkey@0"
I20260807 20:12:10.239799 24635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "2fe67533e43d4624bb16596401a1fb95"
format_stamp: "Formatted at 2026-08-07 20:12:10 on dist-test-slave-1x4g"
server_key: "6dac2d6df6fb8de2858c681158716396"
server_key_iv: "aadd2bf9d85ad5dbec9c27ec906e4564"
server_key_version: "encryptionkey@0"
I20260807 20:12:10.241295 24635 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:10.242095 24651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:10.242275 24635 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:10.242388 24635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "2fe67533e43d4624bb16596401a1fb95"
format_stamp: "Formatted at 2026-08-07 20:12:10 on dist-test-slave-1x4g"
server_key: "6dac2d6df6fb8de2858c681158716396"
server_key_iv: "aadd2bf9d85ad5dbec9c27ec906e4564"
server_key_version: "encryptionkey@0"
I20260807 20:12:10.242465 24635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:10.257802 24635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:10.259666 24635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:10.259835 24635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:10.264221 24635 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34697
I20260807 20:12:10.264256 24705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34697 every 8 connection(s)
I20260807 20:12:10.264657 24635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:10.265789 24706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:10.268164 24706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95: Bootstrap starting.
I20260807 20:12:10.268726 24706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:10.269047 24706 log.cc:824] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:10.269754 24706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95: No bootstrap required, opened a new log
I20260807 20:12:10.271301 24706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fe67533e43d4624bb16596401a1fb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34697 } }
I20260807 20:12:10.271441 24706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:10.271463 24706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fe67533e43d4624bb16596401a1fb95, State: Initialized, Role: FOLLOWER
I20260807 20:12:10.271574 24706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fe67533e43d4624bb16596401a1fb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34697 } }
I20260807 20:12:10.271658 24706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:10.271693 24706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:10.271749 24706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:10.272296 24706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fe67533e43d4624bb16596401a1fb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34697 } }
I20260807 20:12:10.272420 24706 leader_election.cc:302] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fe67533e43d4624bb16596401a1fb95; no voters:
I20260807 20:12:10.272611 24706 leader_election.cc:288] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:10.272661 24709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:10.272854 24709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [term 1 LEADER]: Becoming Leader. State: Replica: 2fe67533e43d4624bb16596401a1fb95, State: Running, Role: LEADER
I20260807 20:12:10.272905 24706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:10.272998 24709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fe67533e43d4624bb16596401a1fb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34697 } }
I20260807 20:12:10.273564 24710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fe67533e43d4624bb16596401a1fb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fe67533e43d4624bb16596401a1fb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34697 } } }
I20260807 20:12:10.273684 24711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fe67533e43d4624bb16596401a1fb95. Latest consensus state: current_term: 1 leader_uuid: "2fe67533e43d4624bb16596401a1fb95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fe67533e43d4624bb16596401a1fb95" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34697 } } }
I20260807 20:12:10.273738 24710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:10.273770 24711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:10.274077 24718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:10.274724 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24635
I20260807 20:12:10.274853 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:10.275066 3134 external_mini_cluster.cc:1428] Setting key 47860747dcd1a7c8afa6423b725b49bc
I20260807 20:12:10.274699 24718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:10.277253 24718 catalog_manager.cc:1347] Generated new cluster ID: e85223ae9998421281c62dcc4594cfe8
I20260807 20:12:10.277309 24718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:10.281256 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:10.275684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59201 (local address 127.0.0.1:34697)
0807 20:12:10.275852 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:10.275861 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:10.275893 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:12:10.276022 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:10.276029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:10.276097 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:10.276244 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:10.276505 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:10.277827 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:10.278888 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:10.279477 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:10.279570 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:10.280486 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:10.280490 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:10.280490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:10.280687 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:10.280723 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:10.280737 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:10.280870 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:10.281019 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 20:12:10.281048 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260807 20:12:10.282229 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:10.285871 24718 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:10.286406 24718 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:10.287282 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:10.283684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34941 (local address 127.0.0.1:34697)
0807 20:12:10.283719 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:10.283721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:10.283835 (+ 114us) server_negotiation.cc:407] Connection header received
0807 20:12:10.283895 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:10.283896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:10.283906 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:10.283953 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:10.284087 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:10.284821 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:10.285894 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:10.286233 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:10.286299 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:10.287020 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:10.287022 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:10.287022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:10.287026 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:10.287044 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:10.287048 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:10.287120 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:10.287194 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:12:10.287207 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:12:10.291435 24718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fe67533e43d4624bb16596401a1fb95: Generated new TSK 0
I20260807 20:12:10.291615 24718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:10.294392 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24635
W20260807 20:12:10.300307 24618 connection.cc:535] client connection to 127.0.0.1:34697 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34697): Connection reset by peer (error 104)
2026-08-07T20:12:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1348 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:10 dist-test-slave-1x4g krb5kdc[24738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:10 dist-test-slave-1x4g krb5kdc[24738](info): set up 2 sockets
Aug 07 20:12:10 dist-test-slave-1x4g krb5kdc[24738](info): commencing operation
krb5kdc: starting...
W20260807 20:12:11.433557 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:11 dist-test-slave-1x4g krb5kdc[24738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133531, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:11.556941 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38885
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:11.641012 24751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:11.641214 24751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:11.641237 24751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:11.642413 24751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:11.642457 24751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:11.642472 24751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:11.642509 24751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:11.642539 24751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:11.644009 24751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38885
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:11.644268 24751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:11.644488 24751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:11.649438 24757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:11.649431 24756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:11.649616 24751 server_base.cc:1034] running on GCE node
W20260807 20:12:11.649456 24760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:11.649974 24751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:11.650264 24751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:11.651428 24751 hybrid_clock.cc:648] HybridClock initialized: now 1786133531651421 us; error 37 us; skew 500 ppm
Aug 07 20:12:11 dist-test-slave-1x4g krb5kdc[24738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:11.654011 24751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:11.654390 24751 webserver.cc:460] Webserver started at http://127.0.0.1:45793/ using document root <none> and password file <none>
I20260807 20:12:11.654596 24751 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:11.654637 24751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:11.654726 24751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:11.655702 24751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "bddce4f049cc4c7fa27768fa3715af8e"
format_stamp: "Formatted at 2026-08-07 20:12:11 on dist-test-slave-1x4g"
server_key: "fe6ed70782b330a749d68787514f22e5"
server_key_iv: "5ebfa5c432f208c555e2ec06a0cd4389"
server_key_version: "encryptionkey@0"
I20260807 20:12:11.655993 24751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "bddce4f049cc4c7fa27768fa3715af8e"
format_stamp: "Formatted at 2026-08-07 20:12:11 on dist-test-slave-1x4g"
server_key: "fe6ed70782b330a749d68787514f22e5"
server_key_iv: "5ebfa5c432f208c555e2ec06a0cd4389"
server_key_version: "encryptionkey@0"
I20260807 20:12:11.657356 24751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:11.658043 24768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:11.658241 24751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:11.658329 24751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "bddce4f049cc4c7fa27768fa3715af8e"
format_stamp: "Formatted at 2026-08-07 20:12:11 on dist-test-slave-1x4g"
server_key: "fe6ed70782b330a749d68787514f22e5"
server_key_iv: "5ebfa5c432f208c555e2ec06a0cd4389"
server_key_version: "encryptionkey@0"
I20260807 20:12:11.658385 24751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:11.661777 24751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:11.663542 24751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:11.663647 24751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:11.667644 24751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38885
I20260807 20:12:11.667711 24823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38885 every 8 connection(s)
I20260807 20:12:11.668041 24751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:11.669092 24824 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:11.671312 24824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e: Bootstrap starting.
I20260807 20:12:11.671886 24824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:11.671949 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24751
I20260807 20:12:11.672115 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:11.672264 24824 log.cc:824] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:11.672322 3134 external_mini_cluster.cc:1428] Setting key d444fd2da8991a8d63fcadad7b6508cf
I20260807 20:12:11.673070 24824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e: No bootstrap required, opened a new log
I20260807 20:12:11.674764 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:11.672825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47949 (local address 127.0.0.1:38885)
0807 20:12:11.673007 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:11.673017 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:11.673050 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:11.673519 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:11.673526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:11.673551 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:11.673696 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:11.674365 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:11.674369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:11.674369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:11.674373 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:11.674421 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:11.674431 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:11.674528 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:11.674561 (+ 33us) server_negotiation.cc:299] Negotiation successful
0807 20:12:11.674602 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260807 20:12:11.675662 24824 raft_consensus.cc:348] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddce4f049cc4c7fa27768fa3715af8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38885 } }
I20260807 20:12:11.675812 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:11.675871 24824 raft_consensus.cc:374] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:11.675923 24824 raft_consensus.cc:729] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bddce4f049cc4c7fa27768fa3715af8e, State: Initialized, Role: FOLLOWER
I20260807 20:12:11.676043 24824 consensus_queue.cc:260] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddce4f049cc4c7fa27768fa3715af8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38885 } }
I20260807 20:12:11.676168 24824 raft_consensus.cc:388] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:11.676215 24824 raft_consensus.cc:482] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:11.676266 24824 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:11.676877 24824 raft_consensus.cc:504] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddce4f049cc4c7fa27768fa3715af8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38885 } }
I20260807 20:12:11.677003 24824 leader_election.cc:302] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bddce4f049cc4c7fa27768fa3715af8e; no voters:
I20260807 20:12:11.677170 24824 leader_election.cc:288] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:11.677199 24833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:11.677392 24833 raft_consensus.cc:686] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [term 1 LEADER]: Becoming Leader. State: Replica: bddce4f049cc4c7fa27768fa3715af8e, State: Running, Role: LEADER
I20260807 20:12:11.677430 24824 sys_catalog.cc:564] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:11.677659 24833 consensus_queue.cc:237] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddce4f049cc4c7fa27768fa3715af8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38885 } }
I20260807 20:12:11.678254 24836 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bddce4f049cc4c7fa27768fa3715af8e. Latest consensus state: current_term: 1 leader_uuid: "bddce4f049cc4c7fa27768fa3715af8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddce4f049cc4c7fa27768fa3715af8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38885 } } }
I20260807 20:12:11.678247 24834 sys_catalog.cc:455] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bddce4f049cc4c7fa27768fa3715af8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bddce4f049cc4c7fa27768fa3715af8e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38885 } } }
I20260807 20:12:11.678416 24836 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:11.678479 24834 sys_catalog.cc:458] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:11.678730 24843 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:11.679005 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:11.677437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56753 (local address 127.0.0.1:38885)
0807 20:12:11.677473 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:11.677475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:11.677686 (+ 211us) server_negotiation.cc:407] Connection header received
0807 20:12:11.677777 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:11.677778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:11.677783 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:11.677847 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:11.678616 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:11.678618 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:11.678618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:11.678623 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:11.678639 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:11.678644 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:11.678705 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:11.678928 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 20:12:11.678942 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:12:11.679656 24843 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:11.681106 24843 catalog_manager.cc:1347] Generated new cluster ID: 236b1d8e58154e3eb8aca5c2d7cc4ee9
I20260807 20:12:11.681157 24843 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:11.688342 24843 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:11.688495 24843 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:11.691522 24843 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bddce4f049cc4c7fa27768fa3715af8e: Generated new TSK 0
I20260807 20:12:11.691709 24843 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:11.698220 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24751
2026-08-07T20:12:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:11 dist-test-slave-1x4g krb5kdc[24856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:11 dist-test-slave-1x4g krb5kdc[24856](info): set up 2 sockets
Aug 07 20:12:11 dist-test-slave-1x4g krb5kdc[24856](info): commencing operation
krb5kdc: starting...
W20260807 20:12:12.782198 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:12 dist-test-slave-1x4g krb5kdc[24856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:12.907961 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43583
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:12.992362 24869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:12.992569 24869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:12.992592 24869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:12.993678 24869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:12.993719 24869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:12.993733 24869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:12.993745 24869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:12.993757 24869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:12.995185 24869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:12.995390 24869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:12.995566 24869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:13.000577 24875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:13.000594 24877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:13.000684 24869 server_base.cc:1034] running on GCE node
W20260807 20:12:13.000569 24874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:13.001035 24869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:13.001365 24869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:13.002542 24869 hybrid_clock.cc:648] HybridClock initialized: now 1786133533002533 us; error 42 us; skew 500 ppm
Aug 07 20:12:12 dist-test-slave-1x4g krb5kdc[24856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133532, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:13.005079 24869 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:13.005574 24869 webserver.cc:460] Webserver started at http://127.0.0.1:40509/ using document root <none> and password file <none>
I20260807 20:12:13.005906 24869 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:13.005980 24869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:13.006134 24869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:13.007215 24869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "db4200526db548f09d649b47e925f3ed"
format_stamp: "Formatted at 2026-08-07 20:12:13 on dist-test-slave-1x4g"
server_key: "bed235d292509fb407586dbad638ffc4"
server_key_iv: "6fbe39b2c8ffd11f123b418174967752"
server_key_version: "encryptionkey@0"
I20260807 20:12:13.007612 24869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "db4200526db548f09d649b47e925f3ed"
format_stamp: "Formatted at 2026-08-07 20:12:13 on dist-test-slave-1x4g"
server_key: "bed235d292509fb407586dbad638ffc4"
server_key_iv: "6fbe39b2c8ffd11f123b418174967752"
server_key_version: "encryptionkey@0"
I20260807 20:12:13.009050 24869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 20:12:13.009809 24886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:13.009979 24869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:13.010087 24869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "db4200526db548f09d649b47e925f3ed"
format_stamp: "Formatted at 2026-08-07 20:12:13 on dist-test-slave-1x4g"
server_key: "bed235d292509fb407586dbad638ffc4"
server_key_iv: "6fbe39b2c8ffd11f123b418174967752"
server_key_version: "encryptionkey@0"
I20260807 20:12:13.010169 24869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:13.036741 24869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:13.038853 24869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:13.039038 24869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:13.043330 24869 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43583
I20260807 20:12:13.043367 24941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43583 every 8 connection(s)
I20260807 20:12:13.043781 24869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:13.044904 24942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:13.047235 24942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed: Bootstrap starting.
I20260807 20:12:13.047778 24942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:13.048030 24942 log.cc:824] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:13.048722 24942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed: No bootstrap required, opened a new log
I20260807 20:12:13.050323 24942 raft_consensus.cc:348] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4200526db548f09d649b47e925f3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43583 } }
I20260807 20:12:13.050465 24942 raft_consensus.cc:374] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:13.050487 24942 raft_consensus.cc:729] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db4200526db548f09d649b47e925f3ed, State: Initialized, Role: FOLLOWER
I20260807 20:12:13.050606 24942 consensus_queue.cc:260] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4200526db548f09d649b47e925f3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43583 } }
I20260807 20:12:13.050657 24942 raft_consensus.cc:388] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:13.050678 24942 raft_consensus.cc:482] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:13.050705 24942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:13.051317 24942 raft_consensus.cc:504] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4200526db548f09d649b47e925f3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43583 } }
I20260807 20:12:13.051407 24942 leader_election.cc:302] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db4200526db548f09d649b47e925f3ed; no voters:
I20260807 20:12:13.051530 24942 leader_election.cc:288] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:13.051616 24945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:13.051789 24945 raft_consensus.cc:686] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [term 1 LEADER]: Becoming Leader. State: Replica: db4200526db548f09d649b47e925f3ed, State: Running, Role: LEADER
I20260807 20:12:13.051823 24942 sys_catalog.cc:564] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:13.051882 24945 consensus_queue.cc:237] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4200526db548f09d649b47e925f3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43583 } }
I20260807 20:12:13.052279 24947 sys_catalog.cc:455] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader db4200526db548f09d649b47e925f3ed. Latest consensus state: current_term: 1 leader_uuid: "db4200526db548f09d649b47e925f3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4200526db548f09d649b47e925f3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43583 } } }
I20260807 20:12:13.052376 24947 sys_catalog.cc:458] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:13.052386 24946 sys_catalog.cc:455] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db4200526db548f09d649b47e925f3ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db4200526db548f09d649b47e925f3ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43583 } } }
I20260807 20:12:13.052470 24946 sys_catalog.cc:458] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:13.052613 24954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:13.053195 24954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:13.053761 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24869
I20260807 20:12:13.053885 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:13.054051 3134 external_mini_cluster.cc:1428] Setting key 94f81ff8b87ab59e2d724790fc12d5ee
I20260807 20:12:13.055953 24954 catalog_manager.cc:1347] Generated new cluster ID: dd7b631690cf480caef0f4e3f9b9464b
I20260807 20:12:13.056015 24954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:13 dist-test-slave-1x4g krb5kdc[24856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133532, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:12:13.071159 24954 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:13.071841 24954 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:13.075251 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:13.055975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35037 (local address 127.0.0.1:43583)
0807 20:12:13.063330 (+ 7355us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:13.063342 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:13.063395 (+ 53us) server_negotiation.cc:407] Connection header received
0807 20:12:13.063542 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:13.063550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:13.063611 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:13.063776 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:13.064561 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:13.066122 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:13.067170 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:13.067836 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:13.067961 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:13.070958 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:13.070963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:13.070964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:13.070978 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:13.073206 (+ 2228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:13.073617 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:13.073620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:13.073621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:13.073729 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:13.073928 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:13.073929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:13.073929 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:13.074224 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:13.074386 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:13.074853 (+ 467us) server_negotiation.cc:299] Negotiation successful
0807 20:12:13.075006 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7295,"thread_start_us":71,"threads_started":1}
I20260807 20:12:13.076184 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:13.085101 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:13.077819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36315 (local address 127.0.0.1:43583)
0807 20:12:13.077859 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:13.077861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:13.077906 (+ 45us) server_negotiation.cc:407] Connection header received
0807 20:12:13.077972 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:13.077973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:13.077983 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:13.078041 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:13.078656 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:13.079448 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:13.080499 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:13.080853 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:13.080919 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:13.082276 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:13.082277 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:13.082277 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:13.082285 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:13.083903 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:13.084277 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:13.084278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:13.084278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:13.084342 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:13.084541 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:13.084542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:13.084542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:13.084672 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:13.084814 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:13.084917 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 20:12:13.085004 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:13.087232 24954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db4200526db548f09d649b47e925f3ed: Generated new TSK 0
I20260807 20:12:13.087392 24954 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:13.094858 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24869
2026-08-07T20:12:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1397 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:13 dist-test-slave-1x4g krb5kdc[24975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:13 dist-test-slave-1x4g krb5kdc[24975](info): set up 2 sockets
Aug 07 20:12:13 dist-test-slave-1x4g krb5kdc[24975](info): commencing operation
krb5kdc: starting...
W20260807 20:12:14.157698 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:14 dist-test-slave-1x4g krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133534, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:14.283722 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:14.366961 24989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:14.367183 24989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:14.367206 24989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:14.368306 24989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:14.368342 24989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:14.368357 24989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:14.368369 24989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:14.368381 24989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:14.369825 24989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:14.370025 24989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:14.370198 24989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:14.375341 24994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:14.375504 24997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:14.375377 24989 server_base.cc:1034] running on GCE node
W20260807 20:12:14.375379 24995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:14.375972 24989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:14.376363 24989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:14.377540 24989 hybrid_clock.cc:648] HybridClock initialized: now 1786133534377514 us; error 42 us; skew 500 ppm
Aug 07 20:12:14 dist-test-slave-1x4g krb5kdc[24975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133534, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:14.384721 24989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:14.385275 24989 webserver.cc:460] Webserver started at http://127.0.0.1:38665/ using document root <none> and password file <none>
I20260807 20:12:14.385526 24989 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:14.385576 24989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:14.385751 24989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:14.386861 24989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "fd8f45ec02fd4abfb189a18924f9ac04"
format_stamp: "Formatted at 2026-08-07 20:12:14 on dist-test-slave-1x4g"
server_key: "106f3cc18ca71a6636ed44764ef4e789"
server_key_iv: "c08b8fa6e8fd54bbb3803b341cf41f04"
server_key_version: "encryptionkey@0"
I20260807 20:12:14.387239 24989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "fd8f45ec02fd4abfb189a18924f9ac04"
format_stamp: "Formatted at 2026-08-07 20:12:14 on dist-test-slave-1x4g"
server_key: "106f3cc18ca71a6636ed44764ef4e789"
server_key_iv: "c08b8fa6e8fd54bbb3803b341cf41f04"
server_key_version: "encryptionkey@0"
I20260807 20:12:14.388584 24989 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 20:12:14.389408 25004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:14.389612 24989 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:14.389719 24989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "fd8f45ec02fd4abfb189a18924f9ac04"
format_stamp: "Formatted at 2026-08-07 20:12:14 on dist-test-slave-1x4g"
server_key: "106f3cc18ca71a6636ed44764ef4e789"
server_key_iv: "c08b8fa6e8fd54bbb3803b341cf41f04"
server_key_version: "encryptionkey@0"
I20260807 20:12:14.389792 24989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:14.401998 24989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:14.403961 24989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:14.404121 24989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:14.408198 24989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39373
I20260807 20:12:14.408247 25062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39373 every 8 connection(s)
I20260807 20:12:14.408591 24989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:14.408854 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 24989
I20260807 20:12:14.409003 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:14.409209 3134 external_mini_cluster.cc:1428] Setting key 3a4516eba68d304c1cc76e5c64decda3
I20260807 20:12:14.410023 25063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:14.413291 25063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04: Bootstrap starting.
I20260807 20:12:14.414019 25063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:14.414369 25063 log.cc:824] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:14.415226 25063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04: No bootstrap required, opened a new log
I20260807 20:12:14.415522 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:14.409803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52033 (local address 127.0.0.1:39373)
0807 20:12:14.410015 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:14.410026 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:14.410087 (+ 61us) server_negotiation.cc:407] Connection header received
0807 20:12:14.410728 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:14.410738 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:14.410805 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:14.410963 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:14.411143 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:14.412522 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:14.413372 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:14.413994 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:14.414117 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:14.414876 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:14.414881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:14.414881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:14.415055 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:14.415087 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:14.415098 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:14.415150 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:14.415258 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:12:14.415349 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":100,"threads_started":1}
I20260807 20:12:14.416572 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:14.417624 25063 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260807 20:12:14.417766 25063 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:14.417819 25063 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8f45ec02fd4abfb189a18924f9ac04, State: Initialized, Role: FOLLOWER
I20260807 20:12:14.417949 25063 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260807 20:12:14.418068 25063 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:14.418129 25063 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:14.418195 25063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:14.419003 25063 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260807 20:12:14.419127 25063 leader_election.cc:302] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd8f45ec02fd4abfb189a18924f9ac04; no voters:
I20260807 20:12:14.419305 25063 leader_election.cc:288] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:14.419375 25074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:14.419574 25063 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:14.419612 25074 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [term 1 LEADER]: Becoming Leader. State: Replica: fd8f45ec02fd4abfb189a18924f9ac04, State: Running, Role: LEADER
I20260807 20:12:14.419879 25074 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } }
I20260807 20:12:14.420231 25075 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } } }
I20260807 20:12:14.420320 25075 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:14.420483 25076 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8f45ec02fd4abfb189a18924f9ac04. Latest consensus state: current_term: 1 leader_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8f45ec02fd4abfb189a18924f9ac04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39373 } } }
I20260807 20:12:14.420534 25076 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:14.420727 25083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:14.421741 25083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:14.421882 25067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:14.417999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58799 (local address 127.0.0.1:39373)
0807 20:12:14.418038 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:14.418040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:14.418274 (+ 234us) server_negotiation.cc:407] Connection header received
0807 20:12:14.418297 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:14.418298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:14.418309 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:14.418360 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:14.418497 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:14.419219 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:14.420304 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:14.420643 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:14.420716 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:14.421487 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:14.421488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:14.421489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:14.421494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:14.421510 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:14.421514 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:14.421558 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:14.421790 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 20:12:14.421820 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:14.429984 25083 catalog_manager.cc:1347] Generated new cluster ID: 2f3ba44cb5794442b6f1f900b248bed0
I20260807 20:12:14.430054 25083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:14.443987 25083 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:14.444870 25083 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:14.449383 25083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd8f45ec02fd4abfb189a18924f9ac04: Generated new TSK 0
I20260807 20:12:14.449582 25083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:14.457818 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 24989
2026-08-07T20:12:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1362 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:14 dist-test-slave-1x4g krb5kdc[25097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:14 dist-test-slave-1x4g krb5kdc[25097](info): set up 2 sockets
Aug 07 20:12:14 dist-test-slave-1x4g krb5kdc[25097](info): commencing operation
krb5kdc: starting...
W20260807 20:12:15.533653 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:15 dist-test-slave-1x4g krb5kdc[25097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:15.659868 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45111
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:15.741950 25113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:15.742209 25113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:15.742249 25113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:15.743388 25113 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:15.743461 25113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:15.743486 25113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:15.743515 25113 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:15.743544 25113 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:15.745074 25113 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25113
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:15.745354 25113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:15.745581 25113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:15.750469 25118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:15.750651 25122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:15.750536 25113 server_base.cc:1034] running on GCE node
W20260807 20:12:15.750520 25120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:15.751004 25113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:15.751313 25113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:15.752462 25113 hybrid_clock.cc:648] HybridClock initialized: now 1786133535752442 us; error 37 us; skew 500 ppm
Aug 07 20:12:15 dist-test-slave-1x4g krb5kdc[25097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133535, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:15.755009 25113 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:15.755527 25113 webserver.cc:460] Webserver started at http://127.0.0.1:41685/ using document root <none> and password file <none>
I20260807 20:12:15.755756 25113 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:15.755801 25113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:15.755971 25113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:15.757145 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "3de33200bbf64940b0d6d3bca7b86132"
format_stamp: "Formatted at 2026-08-07 20:12:15 on dist-test-slave-1x4g"
server_key: "d219b78066736f59b4a3914923626b5f"
server_key_iv: "0077c50c50396f439fecafefd087b024"
server_key_version: "encryptionkey@0"
I20260807 20:12:15.757514 25113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "3de33200bbf64940b0d6d3bca7b86132"
format_stamp: "Formatted at 2026-08-07 20:12:15 on dist-test-slave-1x4g"
server_key: "d219b78066736f59b4a3914923626b5f"
server_key_iv: "0077c50c50396f439fecafefd087b024"
server_key_version: "encryptionkey@0"
I20260807 20:12:15.758855 25113 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 20:12:15.759606 25130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:15.759786 25113 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:15.759883 25113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "3de33200bbf64940b0d6d3bca7b86132"
format_stamp: "Formatted at 2026-08-07 20:12:15 on dist-test-slave-1x4g"
server_key: "d219b78066736f59b4a3914923626b5f"
server_key_iv: "0077c50c50396f439fecafefd087b024"
server_key_version: "encryptionkey@0"
I20260807 20:12:15.759954 25113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:15.763353 25113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:15.765375 25113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:15.765530 25113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:15.769629 25113 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45111
I20260807 20:12:15.769659 25185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45111 every 8 connection(s)
I20260807 20:12:15.769979 25113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:15.770946 25186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:15.773412 25186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132: Bootstrap starting.
I20260807 20:12:15.774011 25186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:15.774317 25186 log.cc:824] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:15.775045 25186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132: No bootstrap required, opened a new log
I20260807 20:12:15.775219 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25113
I20260807 20:12:15.775343 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:15.775525 3134 external_mini_cluster.cc:1428] Setting key f8339daa4c5945739e89bb6309484175
I20260807 20:12:15.777127 25186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de33200bbf64940b0d6d3bca7b86132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45111 } }
I20260807 20:12:15.777262 25186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:15.777284 25186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de33200bbf64940b0d6d3bca7b86132, State: Initialized, Role: FOLLOWER
I20260807 20:12:15.777544 25186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de33200bbf64940b0d6d3bca7b86132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45111 } }
I20260807 20:12:15.777659 25186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:15.777696 25186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:15.777748 25186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:15.778698 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:15.776046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41843 (local address 127.0.0.1:45111)
0807 20:12:15.776239 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:15.776248 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:15.776280 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:12:15.777144 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:15.777154 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:15.777194 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:15.777348 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:15.778153 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:15.778157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:15.778158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:15.778161 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:15.778197 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:15.778232 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:15.778300 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:15.778427 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 20:12:15.778460 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260807 20:12:15.778954 25186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de33200bbf64940b0d6d3bca7b86132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45111 } }
I20260807 20:12:15.779069 25186 leader_election.cc:302] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3de33200bbf64940b0d6d3bca7b86132; no voters:
I20260807 20:12:15.779260 25186 leader_election.cc:288] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:15.779309 25191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:15.779457 25191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [term 1 LEADER]: Becoming Leader. State: Replica: 3de33200bbf64940b0d6d3bca7b86132, State: Running, Role: LEADER
I20260807 20:12:15.779565 25186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:15.779613 25191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de33200bbf64940b0d6d3bca7b86132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45111 } }
I20260807 20:12:15.779811 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:15.780910 25192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3de33200bbf64940b0d6d3bca7b86132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de33200bbf64940b0d6d3bca7b86132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45111 } } }
I20260807 20:12:15.781008 25192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:15.781603 25208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 20:12:15.781436 25194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3de33200bbf64940b0d6d3bca7b86132. Latest consensus state: current_term: 1 leader_uuid: "3de33200bbf64940b0d6d3bca7b86132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de33200bbf64940b0d6d3bca7b86132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45111 } } }
W20260807 20:12:15.781672 25208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:15.781688 25194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:15.781764 25211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:15.781950 25211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:15.783596 25211 catalog_manager.cc:1347] Generated new cluster ID: 76673b1625cd4e5180320e7c9f63a91a
I20260807 20:12:15.783659 25211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:15.783723 25190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:15.781465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47403 (local address 127.0.0.1:45111)
0807 20:12:15.781509 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:15.781510 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:15.782552 (+ 1042us) server_negotiation.cc:407] Connection header received
0807 20:12:15.782661 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:15.782662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:15.782667 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:15.782695 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:15.783386 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:15.783387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:15.783387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:15.783391 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:15.783405 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:15.783409 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:15.783439 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:15.783545 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 20:12:15.783562 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 20:12:15.803092 25211 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:15.803318 25211 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:15.812387 25211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3de33200bbf64940b0d6d3bca7b86132: Generated new TSK 0
I20260807 20:12:15.812624 25211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:15.816838 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25113
2026-08-07T20:12:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1358 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:15 dist-test-slave-1x4g krb5kdc[25219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:15 dist-test-slave-1x4g krb5kdc[25219](info): set up 2 sockets
Aug 07 20:12:15 dist-test-slave-1x4g krb5kdc[25219](info): commencing operation
krb5kdc: starting...
W20260807 20:12:16.886021 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:16 dist-test-slave-1x4g krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:17.011708 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:17.094611 25232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:17.094816 25232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:17.094838 25232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:17.095991 25232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:17.096035 25232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:17.096050 25232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:17.096092 25232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:17.096123 25232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:17.097623 25232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:17.097886 25232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:17.098129 25232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:17.103001 25237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:17.103001 25238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:17.103258 25232 server_base.cc:1034] running on GCE node
W20260807 20:12:17.103071 25240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:17.103528 25232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:17.103813 25232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:17.104977 25232 hybrid_clock.cc:648] HybridClock initialized: now 1786133537104971 us; error 35 us; skew 500 ppm
Aug 07 20:12:17 dist-test-slave-1x4g krb5kdc[25219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133537, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:17.107443 25232 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:17.107890 25232 webserver.cc:460] Webserver started at http://127.0.0.1:36009/ using document root <none> and password file <none>
I20260807 20:12:17.108124 25232 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:17.108196 25232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:17.108347 25232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:17.109485 25232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "dc912393ab7a452f918edaafd13d218e"
format_stamp: "Formatted at 2026-08-07 20:12:17 on dist-test-slave-1x4g"
server_key: "5f5cae92632e00a33b950598b1ecfdaa"
server_key_iv: "040270406fbcad087e51c53ebf50daf5"
server_key_version: "encryptionkey@0"
I20260807 20:12:17.109876 25232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "dc912393ab7a452f918edaafd13d218e"
format_stamp: "Formatted at 2026-08-07 20:12:17 on dist-test-slave-1x4g"
server_key: "5f5cae92632e00a33b950598b1ecfdaa"
server_key_iv: "040270406fbcad087e51c53ebf50daf5"
server_key_version: "encryptionkey@0"
I20260807 20:12:17.111230 25232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:17.111970 25247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:17.112144 25232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:17.112254 25232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "dc912393ab7a452f918edaafd13d218e"
format_stamp: "Formatted at 2026-08-07 20:12:17 on dist-test-slave-1x4g"
server_key: "5f5cae92632e00a33b950598b1ecfdaa"
server_key_iv: "040270406fbcad087e51c53ebf50daf5"
server_key_version: "encryptionkey@0"
I20260807 20:12:17.112332 25232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:17.119920 25232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:17.121879 25232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:17.122045 25232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:17.126155 25232 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41245
I20260807 20:12:17.126220 25304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41245 every 8 connection(s)
I20260807 20:12:17.126590 25232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:17.127132 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25232
I20260807 20:12:17.127264 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:17.127523 3134 external_mini_cluster.cc:1428] Setting key 757684b849042a8911bf2fb29bc6d780
I20260807 20:12:17.127867 25305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:17.131291 25305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e: Bootstrap starting.
I20260807 20:12:17.132025 25305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:17.132367 25305 log.cc:824] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:17.133216 25305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e: No bootstrap required, opened a new log
Aug 07 20:12:17 dist-test-slave-1x4g krb5kdc[25219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:12:17.135511 25305 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc912393ab7a452f918edaafd13d218e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41245 } }
I20260807 20:12:17.135677 25305 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:17.135703 25305 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc912393ab7a452f918edaafd13d218e, State: Initialized, Role: FOLLOWER
I20260807 20:12:17.135859 25305 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc912393ab7a452f918edaafd13d218e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41245 } }
I20260807 20:12:17.135959 25305 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:17.135994 25305 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:17.136044 25305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:17.136770 25305 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc912393ab7a452f918edaafd13d218e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41245 } }
I20260807 20:12:17.136932 25305 leader_election.cc:302] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc912393ab7a452f918edaafd13d218e; no voters:
I20260807 20:12:17.137142 25305 leader_election.cc:288] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:17.137189 25310 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:17.137368 25310 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [term 1 LEADER]: Becoming Leader. State: Replica: dc912393ab7a452f918edaafd13d218e, State: Running, Role: LEADER
I20260807 20:12:17.137470 25305 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:17.137547 25310 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc912393ab7a452f918edaafd13d218e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41245 } }
I20260807 20:12:17.138077 25313 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc912393ab7a452f918edaafd13d218e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc912393ab7a452f918edaafd13d218e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41245 } } }
I20260807 20:12:17.138170 25313 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:17.138387 25310 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc912393ab7a452f918edaafd13d218e. Latest consensus state: current_term: 1 leader_uuid: "dc912393ab7a452f918edaafd13d218e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc912393ab7a452f918edaafd13d218e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41245 } } }
I20260807 20:12:17.138530 25310 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:17.138975 25320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:17.139500 25320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:17.140139 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:17.128167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:52545 (local address 127.0.0.1:41245)
0807 20:12:17.128365 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:17.128373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:17.128400 (+ 27us) server_negotiation.cc:407] Connection header received
0807 20:12:17.129091 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:17.129101 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:17.129160 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:17.129277 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:17.130109 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:17.130984 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:17.132125 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:17.132586 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:17.132712 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:17.135591 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:17.135608 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:17.135608 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:17.135621 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:17.137830 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:17.138372 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:17.138375 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:17.138376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:17.138537 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:17.138856 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:17.138857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:17.138857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:17.139182 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:17.139350 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:17.139766 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 20:12:17.139883 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
I20260807 20:12:17.141108 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:17.141297 25320 catalog_manager.cc:1347] Generated new cluster ID: 64da0aabc1904af8a93212fa1f7f8355
I20260807 20:12:17.141335 25320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:17.148555 25320 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:17.149307 25320 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:17.149981 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:17.142719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:48069 (local address 127.0.0.1:41245)
0807 20:12:17.142753 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:17.142755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:17.142855 (+ 100us) server_negotiation.cc:407] Connection header received
0807 20:12:17.142938 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:17.142939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:17.142949 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:17.142975 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:17.143600 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:17.144335 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:17.145319 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:17.145630 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:17.145665 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:17.147086 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:17.147087 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:17.147088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:17.147105 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:17.148632 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:17.149087 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:17.149088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:17.149088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:17.149155 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:17.149418 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:17.149419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:17.149420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:17.149543 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:17.149615 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:17.149834 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 20:12:17.149897 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 20:12:17.151822 25320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc912393ab7a452f918edaafd13d218e: Generated new TSK 0
I20260807 20:12:17.152012 25320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:17.157683 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25232
W20260807 20:12:17.163941 25217 connection.cc:535] client connection to 127.0.0.1:41245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41245): Connection reset by peer (error 104)
2026-08-07T20:12:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1340 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:17 dist-test-slave-1x4g krb5kdc[25338](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:17 dist-test-slave-1x4g krb5kdc[25338](info): set up 2 sockets
Aug 07 20:12:17 dist-test-slave-1x4g krb5kdc[25338](info): commencing operation
krb5kdc: starting...
W20260807 20:12:18.230428 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:18 dist-test-slave-1x4g krb5kdc[25338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:18.359236 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36219
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36219
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:18.443233 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:18.443436 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:18.443459 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:18.444682 25352 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:18.444726 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:18.444741 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:18.444753 25352 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:18.444834 25352 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:18.446306 25352 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:18.446568 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:18.446794 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:18.451690 25360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:18.451690 25358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:18.451699 25357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:18.451818 25352 server_base.cc:1034] running on GCE node
I20260807 20:12:18.452119 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:18.452471 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:18.453658 25352 hybrid_clock.cc:648] HybridClock initialized: now 1786133538453637 us; error 58 us; skew 500 ppm
Aug 07 20:12:18 dist-test-slave-1x4g krb5kdc[25338](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:18.456212 25352 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:18.456600 25352 webserver.cc:460] Webserver started at http://127.0.0.1:37747/ using document root <none> and password file <none>
I20260807 20:12:18.456881 25352 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:18.456931 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:18.457026 25352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:18.458025 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "6aa3b77f70d14a5aa878418b5fe2192f"
format_stamp: "Formatted at 2026-08-07 20:12:18 on dist-test-slave-1x4g"
server_key: "8deba0b4dd9aff8cde7e7cab03878e1d"
server_key_iv: "2485664df49f71ae896a365dd826457b"
server_key_version: "encryptionkey@0"
I20260807 20:12:18.458338 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "6aa3b77f70d14a5aa878418b5fe2192f"
format_stamp: "Formatted at 2026-08-07 20:12:18 on dist-test-slave-1x4g"
server_key: "8deba0b4dd9aff8cde7e7cab03878e1d"
server_key_iv: "2485664df49f71ae896a365dd826457b"
server_key_version: "encryptionkey@0"
I20260807 20:12:18.459709 25352 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:18.460389 25367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:18.460541 25352 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:18.460621 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "6aa3b77f70d14a5aa878418b5fe2192f"
format_stamp: "Formatted at 2026-08-07 20:12:18 on dist-test-slave-1x4g"
server_key: "8deba0b4dd9aff8cde7e7cab03878e1d"
server_key_iv: "2485664df49f71ae896a365dd826457b"
server_key_version: "encryptionkey@0"
I20260807 20:12:18.460680 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:18.477558 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:18.479598 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:18.479727 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:18.483994 25352 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36219
I20260807 20:12:18.484054 25422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36219 every 8 connection(s)
I20260807 20:12:18.484429 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:18.484692 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25352
I20260807 20:12:18.484853 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:18.485090 3134 external_mini_cluster.cc:1428] Setting key a7c18a9ef7b0d5a6f454568129ada437
I20260807 20:12:18.485738 25424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:18.488754 25424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f: Bootstrap starting.
I20260807 20:12:18.489476 25424 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:18.489866 25424 log.cc:824] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:18.490643 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:18.485635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:37157 (local address 127.0.0.1:36219)
0807 20:12:18.485810 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:18.485836 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:18.485870 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:12:18.486310 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:18.486316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:18.486364 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:18.486476 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:18.486724 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:18.487584 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:18.488691 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:18.489176 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:18.489257 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:18.490026 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:18.490029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:18.490030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:18.490201 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:18.490231 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:18.490241 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:18.490340 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:18.490436 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 20:12:18.490464 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260807 20:12:18.490679 25424 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f: No bootstrap required, opened a new log
I20260807 20:12:18.491648 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:18.493240 25424 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36219 } }
I20260807 20:12:18.493373 25424 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:18.493395 25424 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa3b77f70d14a5aa878418b5fe2192f, State: Initialized, Role: FOLLOWER
I20260807 20:12:18.493542 25424 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36219 } }
I20260807 20:12:18.493618 25424 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:18.493664 25424 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:18.493721 25424 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:18.494246 25424 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36219 } }
I20260807 20:12:18.494362 25424 leader_election.cc:302] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aa3b77f70d14a5aa878418b5fe2192f; no voters:
I20260807 20:12:18.494567 25424 leader_election.cc:288] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:18.494659 25434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:18.494812 25434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [term 1 LEADER]: Becoming Leader. State: Replica: 6aa3b77f70d14a5aa878418b5fe2192f, State: Running, Role: LEADER
I20260807 20:12:18.494872 25424 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:18.494946 25434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36219 } }
I20260807 20:12:18.495275 25434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aa3b77f70d14a5aa878418b5fe2192f. Latest consensus state: current_term: 1 leader_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36219 } } }
I20260807 20:12:18.495339 25434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:18.496150 25437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa3b77f70d14a5aa878418b5fe2192f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36219 } } }
I20260807 20:12:18.496228 25437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:18.496706 25451 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:18.497128 25451 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:18.497252 25440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:18.497458 25440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:18.498404 25427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:18.492984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:55145 (local address 127.0.0.1:36219)
0807 20:12:18.493017 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:18.493019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:18.494768 (+ 1749us) server_negotiation.cc:407] Connection header received
0807 20:12:18.494867 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:18.494868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:18.494880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:18.494909 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:18.495143 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:18.495842 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:18.496974 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:18.497290 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:18.497382 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:18.498150 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:18.498151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:18.498151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:18.498156 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:18.498171 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:18.498175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:18.498229 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:18.498291 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 20:12:18.498300 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:12:18.498989 25440 catalog_manager.cc:1347] Generated new cluster ID: c9a4cf83d0f54e74bc9a288c18e8310a
I20260807 20:12:18.499063 25440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:18.507623 25440 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:18.508236 25440 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:18.519757 25440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aa3b77f70d14a5aa878418b5fe2192f: Generated new TSK 0
I20260807 20:12:18.520002 25440 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:18.524372 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25352
W20260807 20:12:18.531050 25336 connection.cc:535] client connection to 127.0.0.1:36219 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36219): Connection reset by peer (error 104)
2026-08-07T20:12:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1366 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:18 dist-test-slave-1x4g krb5kdc[25458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:18 dist-test-slave-1x4g krb5kdc[25458](info): set up 2 sockets
Aug 07 20:12:18 dist-test-slave-1x4g krb5kdc[25458](info): commencing operation
krb5kdc: starting...
W20260807 20:12:19.586036 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:19 dist-test-slave-1x4g krb5kdc[25458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:19.711336 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40181
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:19.794052 25471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:19.794317 25471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:19.794356 25471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:19.795478 25471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:19.795549 25471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:19.795574 25471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:19.795610 25471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:19.795639 25471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:19.797137 25471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40181
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:19.797422 25471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:19.797648 25471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:12:19.802683 25471 server_base.cc:1034] running on GCE node
W20260807 20:12:19.802632 25477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:19.802673 25478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:19.802635 25480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:19.803078 25471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:19.803386 25471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:19.804540 25471 hybrid_clock.cc:648] HybridClock initialized: now 1786133539804518 us; error 40 us; skew 500 ppm
Aug 07 20:12:19 dist-test-slave-1x4g krb5kdc[25458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133539, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:19.811213 25471 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:19.811695 25471 webserver.cc:460] Webserver started at http://127.0.0.1:34309/ using document root <none> and password file <none>
I20260807 20:12:19.811897 25471 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:19.811937 25471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:19.812031 25471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:19.813160 25471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "ea03c3125f1b453584b43c25612265e3"
format_stamp: "Formatted at 2026-08-07 20:12:19 on dist-test-slave-1x4g"
server_key: "3fe4cef4359db8b72f6c5fefd067f9a0"
server_key_iv: "652a9a7a6c897ea20ee080f44ead199b"
server_key_version: "encryptionkey@0"
I20260807 20:12:19.813494 25471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "ea03c3125f1b453584b43c25612265e3"
format_stamp: "Formatted at 2026-08-07 20:12:19 on dist-test-slave-1x4g"
server_key: "3fe4cef4359db8b72f6c5fefd067f9a0"
server_key_iv: "652a9a7a6c897ea20ee080f44ead199b"
server_key_version: "encryptionkey@0"
I20260807 20:12:19.814793 25471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:19.815490 25488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:19.815678 25471 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:19.815766 25471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "ea03c3125f1b453584b43c25612265e3"
format_stamp: "Formatted at 2026-08-07 20:12:19 on dist-test-slave-1x4g"
server_key: "3fe4cef4359db8b72f6c5fefd067f9a0"
server_key_iv: "652a9a7a6c897ea20ee080f44ead199b"
server_key_version: "encryptionkey@0"
I20260807 20:12:19.815832 25471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:19.819159 25471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:19.821424 25471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:19.821532 25471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:19.825388 25471 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40181
I20260807 20:12:19.825479 25542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40181 every 8 connection(s)
I20260807 20:12:19.825803 25471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:19.826720 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25471
I20260807 20:12:19.826874 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:19.826867 25543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:19.827075 3134 external_mini_cluster.cc:1428] Setting key 15cee4de1fb7929d054675c5fa4dd38a
I20260807 20:12:19.829824 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:19.827638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:38007 (local address 127.0.0.1:40181)
0807 20:12:19.827839 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:19.827848 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:19.827882 (+ 34us) server_negotiation.cc:407] Connection header received
0807 20:12:19.828515 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:19.828525 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:19.828561 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:19.828727 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:19.829380 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:19.829384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:19.829384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:19.829389 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:19.829434 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:19.829447 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:19.829527 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:19.829590 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 20:12:19.829643 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":79,"threads_started":1}
I20260807 20:12:19.830606 25543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3: Bootstrap starting.
I20260807 20:12:19.831389 25543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:19.831740 25543 log.cc:824] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:19.832000 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:19.832463 25543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3: No bootstrap required, opened a new log
I20260807 20:12:19.834554 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:19.833499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:39795 (local address 127.0.0.1:40181)
0807 20:12:19.833537 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:19.833539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:19.833653 (+ 114us) server_negotiation.cc:407] Connection header received
0807 20:12:19.833728 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:19.833729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:19.833734 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:19.833778 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:19.834353 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:19.834354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:19.834354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:19.834359 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:19.834372 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:19.834376 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:19.834424 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:19.834478 (+ 54us) server_negotiation.cc:299] Negotiation successful
0807 20:12:19.834492 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:19.834645 25543 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea03c3125f1b453584b43c25612265e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40181 } }
I20260807 20:12:19.834828 25543 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:19.834865 25543 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea03c3125f1b453584b43c25612265e3, State: Initialized, Role: FOLLOWER
I20260807 20:12:19.834978 25543 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea03c3125f1b453584b43c25612265e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40181 } }
I20260807 20:12:19.835073 25543 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:19.835121 25543 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:19.835165 25543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:19.835652 25543 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea03c3125f1b453584b43c25612265e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40181 } }
I20260807 20:12:19.835740 25543 leader_election.cc:302] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea03c3125f1b453584b43c25612265e3; no voters:
I20260807 20:12:19.835875 25543 leader_election.cc:288] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:19.835952 25555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:19.836133 25543 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:19.836117 25555 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [term 1 LEADER]: Becoming Leader. State: Replica: ea03c3125f1b453584b43c25612265e3, State: Running, Role: LEADER
I20260807 20:12:19.837481 25555 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea03c3125f1b453584b43c25612265e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40181 } }
I20260807 20:12:19.838053 25557 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea03c3125f1b453584b43c25612265e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea03c3125f1b453584b43c25612265e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40181 } } }
I20260807 20:12:19.838286 25557 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:19.838434 25571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:19.838482 25571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:19.838549 25558 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea03c3125f1b453584b43c25612265e3. Latest consensus state: current_term: 1 leader_uuid: "ea03c3125f1b453584b43c25612265e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea03c3125f1b453584b43c25612265e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40181 } } }
I20260807 20:12:19.838578 25573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:19.838653 25558 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:19.838806 25573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:19.840289 25573 catalog_manager.cc:1347] Generated new cluster ID: a67a9f95791f494ab6a1fd74d9a85acb
I20260807 20:12:19.840341 25573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:19.854082 25573 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:19.854254 25573 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:19.856204 25573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea03c3125f1b453584b43c25612265e3: Generated new TSK 0
I20260807 20:12:19.856369 25573 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:19.858700 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25471
2026-08-07T20:12:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:19 dist-test-slave-1x4g krb5kdc[25578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:19 dist-test-slave-1x4g krb5kdc[25578](info): set up 2 sockets
Aug 07 20:12:19 dist-test-slave-1x4g krb5kdc[25578](info): commencing operation
krb5kdc: starting...
W20260807 20:12:20.933779 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:21 dist-test-slave-1x4g krb5kdc[25578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:21.057840 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34899
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:21.141242 25592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:21.141482 25592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:21.141517 25592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:21.142674 25592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:21.142746 25592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:21.142771 25592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:21.142791 25592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:21.142819 25592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:21.144289 25592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34899
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:21.144575 25592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:21.144824 25592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:21.149850 25599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:21.149890 25602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:21.150139 25592 server_base.cc:1034] running on GCE node
W20260807 20:12:21.150261 25600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:21.150502 25592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:21.150807 25592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:21.151955 25592 hybrid_clock.cc:648] HybridClock initialized: now 1786133541151933 us; error 39 us; skew 500 ppm
Aug 07 20:12:21 dist-test-slave-1x4g krb5kdc[25578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133541, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:21.154641 25592 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:21.155117 25592 webserver.cc:460] Webserver started at http://127.0.0.1:33549/ using document root <none> and password file <none>
I20260807 20:12:21.155344 25592 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:21.155411 25592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:21.155567 25592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:21.156672 25592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "09915918fd6047af85f7ca6f68c1a03e"
format_stamp: "Formatted at 2026-08-07 20:12:21 on dist-test-slave-1x4g"
server_key: "59b2b5511e180204234a0fe7b0f56ba7"
server_key_iv: "897eb58903eb2320018cc7a2db5d2c88"
server_key_version: "encryptionkey@0"
I20260807 20:12:21.157125 25592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "09915918fd6047af85f7ca6f68c1a03e"
format_stamp: "Formatted at 2026-08-07 20:12:21 on dist-test-slave-1x4g"
server_key: "59b2b5511e180204234a0fe7b0f56ba7"
server_key_iv: "897eb58903eb2320018cc7a2db5d2c88"
server_key_version: "encryptionkey@0"
I20260807 20:12:21.158505 25592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:21.159289 25609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:21.159490 25592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:21.159613 25592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "09915918fd6047af85f7ca6f68c1a03e"
format_stamp: "Formatted at 2026-08-07 20:12:21 on dist-test-slave-1x4g"
server_key: "59b2b5511e180204234a0fe7b0f56ba7"
server_key_iv: "897eb58903eb2320018cc7a2db5d2c88"
server_key_version: "encryptionkey@0"
I20260807 20:12:21.159693 25592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:21.179708 25592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:21.182243 25592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:21.182420 25592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:21.186964 25592 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34899
I20260807 20:12:21.187036 25664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34899 every 8 connection(s)
I20260807 20:12:21.187420 25592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:21.188493 25665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:21.191438 25665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e: Bootstrap starting.
I20260807 20:12:21.192108 25665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:21.192425 25665 log.cc:824] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:21.193243 25665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e: No bootstrap required, opened a new log
I20260807 20:12:21.193599 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25592
I20260807 20:12:21.193735 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:21.193907 3134 external_mini_cluster.cc:1428] Setting key 73989f7b3432282e096025cd9adf418d
I20260807 20:12:21.195379 25665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09915918fd6047af85f7ca6f68c1a03e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260807 20:12:21.195550 25665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:21.195618 25665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09915918fd6047af85f7ca6f68c1a03e, State: Initialized, Role: FOLLOWER
I20260807 20:12:21.195797 25665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09915918fd6047af85f7ca6f68c1a03e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260807 20:12:21.195894 25665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:21.195948 25665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:21.196000 25665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:21.197005 25665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09915918fd6047af85f7ca6f68c1a03e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260807 20:12:21.197129 25665 leader_election.cc:302] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09915918fd6047af85f7ca6f68c1a03e; no voters:
I20260807 20:12:21.197302 25665 leader_election.cc:288] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:21.197350 25671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:21.197497 25671 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [term 1 LEADER]: Becoming Leader. State: Replica: 09915918fd6047af85f7ca6f68c1a03e, State: Running, Role: LEADER
I20260807 20:12:21.197573 25665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:21.197662 25671 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09915918fd6047af85f7ca6f68c1a03e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } }
I20260807 20:12:21.198031 25671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09915918fd6047af85f7ca6f68c1a03e. Latest consensus state: current_term: 1 leader_uuid: "09915918fd6047af85f7ca6f68c1a03e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09915918fd6047af85f7ca6f68c1a03e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } } }
I20260807 20:12:21.198043 25672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09915918fd6047af85f7ca6f68c1a03e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09915918fd6047af85f7ca6f68c1a03e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34899 } } }
I20260807 20:12:21.198194 25672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:21.198377 25671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:21.198544 25679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:21.199299 25679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:21.201383 25679 catalog_manager.cc:1347] Generated new cluster ID: 5e7cfa71ce764c21b0f3a0314e35d275
I20260807 20:12:21.201438 25679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:21 dist-test-slave-1x4g krb5kdc[25578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 20:12:21.206539 25670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:21.194516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:56877 (local address 127.0.0.1:34899)
0807 20:12:21.194730 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:21.194738 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:21.194768 (+ 30us) server_negotiation.cc:407] Connection header received
0807 20:12:21.195356 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:21.195362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:21.195414 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:21.195555 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:21.196363 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:21.197571 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:21.198712 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:21.199327 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:21.199767 (+ 440us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:21.202424 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:21.202429 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:21.202430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:21.202445 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:21.204496 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:21.204980 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:21.204983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:21.204984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:21.205115 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:21.205391 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:21.205392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:21.205393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:21.205707 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:21.205867 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:21.206053 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 20:12:21.206179 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260807 20:12:21.207520 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:21.216497 25670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:21.209172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:41863 (local address 127.0.0.1:34899)
0807 20:12:21.209252 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:21.209254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:21.209287 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:21.209375 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:21.209376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:21.209385 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:21.209431 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:21.210072 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:21.210806 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:21.211847 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:21.212163 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:21.212223 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:21.213672 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:21.213673 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:21.213673 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:21.213680 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:21.215205 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:21.215676 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:21.215676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:21.215677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:21.215735 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:21.215983 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:21.215984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:21.215984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:21.216097 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:21.216187 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:21.216353 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 20:12:21.216412 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 20:12:21.224468 25679 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:21.225222 25679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:21.235394 25679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09915918fd6047af85f7ca6f68c1a03e: Generated new TSK 0
I20260807 20:12:21.235644 25679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:21.239207 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25592
W20260807 20:12:21.245576 25576 connection.cc:535] client connection to 127.0.0.1:34899 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34899): Connection reset by peer (error 104)
2026-08-07T20:12:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:21 dist-test-slave-1x4g krb5kdc[25697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:21 dist-test-slave-1x4g krb5kdc[25697](info): set up 2 sockets
Aug 07 20:12:21 dist-test-slave-1x4g krb5kdc[25697](info): commencing operation
krb5kdc: starting...
W20260807 20:12:22.357889 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:22 dist-test-slave-1x4g krb5kdc[25697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:22.489377 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37407
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:22.575007 25711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:22.575215 25711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:22.575238 25711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:22.576416 25711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:22.576462 25711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:22.576476 25711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:22.576489 25711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:22.576540 25711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:22.578066 25711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37407
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:22.578320 25711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:22.578553 25711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:22.583758 25717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:22.583739 25716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:22.583814 25711 server_base.cc:1034] running on GCE node
W20260807 20:12:22.583747 25719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:22.584204 25711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:22.584467 25711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:22.585619 25711 hybrid_clock.cc:648] HybridClock initialized: now 1786133542585610 us; error 33 us; skew 500 ppm
Aug 07 20:12:22 dist-test-slave-1x4g krb5kdc[25697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:22.588245 25711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:22.588737 25711 webserver.cc:460] Webserver started at http://127.0.0.1:43539/ using document root <none> and password file <none>
I20260807 20:12:22.588992 25711 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:22.589092 25711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:22.589242 25711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:22.590444 25711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "9ac910f084c24ae49d593f85676976f4"
format_stamp: "Formatted at 2026-08-07 20:12:22 on dist-test-slave-1x4g"
server_key: "ef05ade1c3387635a2cd402d47496db2"
server_key_iv: "990b77966846c349f01afc1f24b3dbef"
server_key_version: "encryptionkey@0"
I20260807 20:12:22.590821 25711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "9ac910f084c24ae49d593f85676976f4"
format_stamp: "Formatted at 2026-08-07 20:12:22 on dist-test-slave-1x4g"
server_key: "ef05ade1c3387635a2cd402d47496db2"
server_key_iv: "990b77966846c349f01afc1f24b3dbef"
server_key_version: "encryptionkey@0"
I20260807 20:12:22.592221 25711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:22.593034 25726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:22.593290 25711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:22.593405 25711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "9ac910f084c24ae49d593f85676976f4"
format_stamp: "Formatted at 2026-08-07 20:12:22 on dist-test-slave-1x4g"
server_key: "ef05ade1c3387635a2cd402d47496db2"
server_key_iv: "990b77966846c349f01afc1f24b3dbef"
server_key_version: "encryptionkey@0"
I20260807 20:12:22.593493 25711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:22.608081 25711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:22.610136 25711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:22.610321 25711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:22.615904 25711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37407
I20260807 20:12:22.615931 25781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37407 every 8 connection(s)
I20260807 20:12:22.616336 25711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:22.617329 25782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:22.619626 25782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4: Bootstrap starting.
I20260807 20:12:22.620177 25782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:22.620419 25782 log.cc:824] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:22.621134 25782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4: No bootstrap required, opened a new log
I20260807 20:12:22.622671 25782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac910f084c24ae49d593f85676976f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37407 } }
I20260807 20:12:22.622807 25782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:22.622830 25782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ac910f084c24ae49d593f85676976f4, State: Initialized, Role: FOLLOWER
I20260807 20:12:22.622933 25782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac910f084c24ae49d593f85676976f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37407 } }
I20260807 20:12:22.623018 25782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:22.623057 25782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:22.623122 25782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:22.623659 25782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac910f084c24ae49d593f85676976f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37407 } }
I20260807 20:12:22.623785 25782 leader_election.cc:302] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ac910f084c24ae49d593f85676976f4; no voters:
I20260807 20:12:22.623970 25782 leader_election.cc:288] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:22.624014 25785 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:22.624193 25785 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [term 1 LEADER]: Becoming Leader. State: Replica: 9ac910f084c24ae49d593f85676976f4, State: Running, Role: LEADER
I20260807 20:12:22.624231 25782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:22.624341 25785 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac910f084c24ae49d593f85676976f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37407 } }
I20260807 20:12:22.624811 25788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ac910f084c24ae49d593f85676976f4. Latest consensus state: current_term: 1 leader_uuid: "9ac910f084c24ae49d593f85676976f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac910f084c24ae49d593f85676976f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37407 } } }
I20260807 20:12:22.624908 25788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:22.624958 25786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ac910f084c24ae49d593f85676976f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ac910f084c24ae49d593f85676976f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37407 } } }
I20260807 20:12:22.625041 25786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:22.625347 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25711
I20260807 20:12:22.625453 25795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:22.625499 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:22.625725 3134 external_mini_cluster.cc:1428] Setting key c52f87cbe9125c1f88e76a076d634798
I20260807 20:12:22.626015 25795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:22.627629 25795 catalog_manager.cc:1347] Generated new cluster ID: 106144fa30874a4e902e70e4ef7b0fc5
I20260807 20:12:22.627684 25795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:22.631313 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:22.626380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:43499 (local address 127.0.0.1:37407)
0807 20:12:22.626737 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:22.626746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:22.626779 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:22.626893 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:22.626901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:22.626959 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:22.627108 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:22.627366 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:22.628439 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:22.629332 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:22.629777 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:22.629904 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:22.630654 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:22.630658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:22.630659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:22.630837 (+ 178us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:22.630867 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:22.630878 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:22.630975 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:22.631059 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 20:12:22.631092 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":69,"threads_started":1}
I20260807 20:12:22.632330 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:22.640200 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:22.637158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:43481 (local address 127.0.0.1:37407)
0807 20:12:22.637198 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:22.637200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:22.637339 (+ 139us) server_negotiation.cc:407] Connection header received
0807 20:12:22.637416 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:22.637417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:22.637430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:22.637507 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:22.637665 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:22.638154 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:22.638907 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:22.639133 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:22.639214 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:22.639915 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:22.639916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:22.639916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:22.639921 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:22.639935 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:22.639939 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:22.640015 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:22.640083 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 20:12:22.640102 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 20:12:22.651788 25795 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:22.652621 25795 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:22.663151 25795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ac910f084c24ae49d593f85676976f4: Generated new TSK 0
I20260807 20:12:22.663383 25795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:22.666352 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25711
W20260807 20:12:22.673055 25695 connection.cc:535] client connection to 127.0.0.1:37407 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37407): Connection reset by peer (error 104)
2026-08-07T20:12:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1427 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:22 dist-test-slave-1x4g krb5kdc[25814](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:22 dist-test-slave-1x4g krb5kdc[25814](info): set up 2 sockets
Aug 07 20:12:22 dist-test-slave-1x4g krb5kdc[25814](info): commencing operation
krb5kdc: starting...
W20260807 20:12:23.773512 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:23 dist-test-slave-1x4g krb5kdc[25814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 20:12:23.898000 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:23.980924 25828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:23.981189 25828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:23.981227 25828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:23.982358 25828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:23.982434 25828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:23.982460 25828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:23.982489 25828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:23.982510 25828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:23.983958 25828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34039
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:23.984233 25828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:23.984454 25828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:23.989337 25836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:23.989326 25834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:23.989389 25833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:23.989614 25828 server_base.cc:1034] running on GCE node
I20260807 20:12:23.989863 25828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:23.990149 25828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:23.991341 25828 hybrid_clock.cc:648] HybridClock initialized: now 1786133543991325 us; error 25 us; skew 500 ppm
Aug 07 20:12:23 dist-test-slave-1x4g krb5kdc[25814](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133543, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:23.993995 25828 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 20:12:23.994446 25828 webserver.cc:460] Webserver started at http://127.0.0.1:42759/ using document root <none> and password file <none>
I20260807 20:12:23.994678 25828 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:23.994750 25828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:23.994910 25828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:23.995975 25828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1"
format_stamp: "Formatted at 2026-08-07 20:12:23 on dist-test-slave-1x4g"
server_key: "b9aec80683b39ee9e7d8e123ac5546b4"
server_key_iv: "cd1ec234ac4da50984208599deec3438"
server_key_version: "encryptionkey@0"
I20260807 20:12:23.996363 25828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1"
format_stamp: "Formatted at 2026-08-07 20:12:23 on dist-test-slave-1x4g"
server_key: "b9aec80683b39ee9e7d8e123ac5546b4"
server_key_iv: "cd1ec234ac4da50984208599deec3438"
server_key_version: "encryptionkey@0"
I20260807 20:12:23.997939 25828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:23.998741 25843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:23.998957 25828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:23.999079 25828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1"
format_stamp: "Formatted at 2026-08-07 20:12:23 on dist-test-slave-1x4g"
server_key: "b9aec80683b39ee9e7d8e123ac5546b4"
server_key_iv: "cd1ec234ac4da50984208599deec3438"
server_key_version: "encryptionkey@0"
I20260807 20:12:23.999168 25828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:24.002563 25828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:24.004444 25828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:24.004608 25828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:24.008616 25828 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34039
I20260807 20:12:24.008684 25897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34039 every 8 connection(s)
I20260807 20:12:24.009089 25828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:24.010144 25898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:24.012405 25898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1: Bootstrap starting.
I20260807 20:12:24.013034 25898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:24.013276 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25828
I20260807 20:12:24.013345 25898 log.cc:824] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:24.013437 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:24.013621 3134 external_mini_cluster.cc:1428] Setting key 9384e22ca999b4c3cdf2cb09867f6c9e
I20260807 20:12:24.014258 25898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1: No bootstrap required, opened a new log
I20260807 20:12:24.016136 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:24.014183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:40815 (local address 127.0.0.1:34039)
0807 20:12:24.014374 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:24.014382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:24.014408 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:12:24.014830 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:24.014836 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:24.014862 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:24.014975 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:24.015664 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:24.015668 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:24.015668 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:24.015672 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:24.015710 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:24.015720 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:24.015787 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:24.015867 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 20:12:24.015930 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260807 20:12:24.017350 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:24.017379 25898 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34039 } }
I20260807 20:12:24.017553 25898 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:24.017585 25898 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad53ebadb764e4aa9ea01f98ccb00c1, State: Initialized, Role: FOLLOWER
I20260807 20:12:24.017712 25898 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34039 } }
I20260807 20:12:24.017795 25898 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:24.017831 25898 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:24.017886 25898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:24.018836 25898 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34039 } }
I20260807 20:12:24.018965 25898 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad53ebadb764e4aa9ea01f98ccb00c1; no voters:
I20260807 20:12:24.019172 25898 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:24.019232 25908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:24.019408 25908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad53ebadb764e4aa9ea01f98ccb00c1, State: Running, Role: LEADER
I20260807 20:12:24.019456 25898 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:24.020767 25902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:24.019143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.163:46543 (local address 127.0.0.1:34039)
0807 20:12:24.019186 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:24.019188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:24.019609 (+ 421us) server_negotiation.cc:407] Connection header received
0807 20:12:24.019690 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:24.019691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:24.019696 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:24.019746 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:24.020455 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:24.020456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:24.020457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:24.020461 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:24.020478 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:24.020483 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:24.020551 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:24.020662 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 20:12:24.020678 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:24.021003 25908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34039 } }
I20260807 20:12:24.021806 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34039 } } }
I20260807 20:12:24.022008 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:24.022071 25926 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:24.022192 25926 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:24.022212 25911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad53ebadb764e4aa9ea01f98ccb00c1. Latest consensus state: current_term: 1 leader_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad53ebadb764e4aa9ea01f98ccb00c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34039 } } }
I20260807 20:12:24.022320 25911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:24.022342 25928 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:24.022529 25928 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:24.024047 25928 catalog_manager.cc:1347] Generated new cluster ID: 40199e23288246d59dbe9c6d4b06b050
I20260807 20:12:24.024101 25928 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:24.041316 25928 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:24.041553 25928 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:24.043730 25928 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad53ebadb764e4aa9ea01f98ccb00c1: Generated new TSK 0
I20260807 20:12:24.043946 25928 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:24.052858 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25828
2026-08-07T20:12:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:24 dist-test-slave-1x4g krb5kdc[25933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:24 dist-test-slave-1x4g krb5kdc[25933](info): set up 2 sockets
Aug 07 20:12:24 dist-test-slave-1x4g krb5kdc[25933](info): commencing operation
krb5kdc: starting...
W20260807 20:12:25.189946 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:25 dist-test-slave-1x4g krb5kdc[25933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:25.315492 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35333
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:39267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:35333
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:25.399362 25946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:25.399710 25946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:25.399752 25946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:25.400934 25946 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:25.401010 25946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:25.401034 25946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:25.401055 25946 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:25.401085 25946 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:25.402628 25946 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:39267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:35333
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:35333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:25.402935 25946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:25.403182 25946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:25.408372 25952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:25.408383 25955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:25.408422 25946 server_base.cc:1034] running on GCE node
W20260807 20:12:25.408416 25953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:25.408855 25946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:25.409209 25946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:25.410445 25946 hybrid_clock.cc:648] HybridClock initialized: now 1786133545410428 us; error 39 us; skew 500 ppm
Aug 07 20:12:25 dist-test-slave-1x4g krb5kdc[25933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133545, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:25.419656 25946 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:25.420136 25946 webserver.cc:460] Webserver started at http://127.3.15.190:39841/ using document root <none> and password file <none>
I20260807 20:12:25.420387 25946 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:25.420460 25946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:25.420625 25946 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:25.421813 25946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "26bc508455ea47df86893d2e34a1d236"
format_stamp: "Formatted at 2026-08-07 20:12:25 on dist-test-slave-1x4g"
server_key: "7346f564ea7cb6d08b4e1d5c152437f1"
server_key_iv: "15dab525515f678165975f6f6d1c17bd"
server_key_version: "encryptionkey@0"
I20260807 20:12:25.422190 25946 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "26bc508455ea47df86893d2e34a1d236"
format_stamp: "Formatted at 2026-08-07 20:12:25 on dist-test-slave-1x4g"
server_key: "7346f564ea7cb6d08b4e1d5c152437f1"
server_key_iv: "15dab525515f678165975f6f6d1c17bd"
server_key_version: "encryptionkey@0"
I20260807 20:12:25.423652 25946 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:25.424438 25962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:25.424691 25946 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:25.424832 25946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "26bc508455ea47df86893d2e34a1d236"
format_stamp: "Formatted at 2026-08-07 20:12:25 on dist-test-slave-1x4g"
server_key: "7346f564ea7cb6d08b4e1d5c152437f1"
server_key_iv: "15dab525515f678165975f6f6d1c17bd"
server_key_version: "encryptionkey@0"
I20260807 20:12:25.424921 25946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:25.455610 25946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:25.457669 25946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:25.457844 25946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:25.462082 25946 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:35333
I20260807 20:12:25.462205 26016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:35333 every 8 connection(s)
I20260807 20:12:25.462534 25946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:25.463551 26017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:25.466184 26017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236: Bootstrap starting.
I20260807 20:12:25.466845 26017 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:25.467185 26017 log.cc:824] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:25.468134 26017 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236: No bootstrap required, opened a new log
I20260807 20:12:25.470391 26017 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bc508455ea47df86893d2e34a1d236" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35333 } }
I20260807 20:12:25.470539 26017 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:25.470562 26017 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26bc508455ea47df86893d2e34a1d236, State: Initialized, Role: FOLLOWER
I20260807 20:12:25.470741 26017 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bc508455ea47df86893d2e34a1d236" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35333 } }
I20260807 20:12:25.470830 26017 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:25.470898 26017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:25.470950 26017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:25.471503 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 25946
I20260807 20:12:25.471665 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:25.471890 3134 external_mini_cluster.cc:1428] Setting key 596cdf4ec0569cfaa16437763f0e1ddb
I20260807 20:12:25.472131 26017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bc508455ea47df86893d2e34a1d236" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35333 } }
I20260807 20:12:25.472272 26017 leader_election.cc:302] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26bc508455ea47df86893d2e34a1d236; no voters:
I20260807 20:12:25.472472 26017 leader_election.cc:288] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:25.472693 26023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:25.472751 26017 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:25.472967 26023 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [term 1 LEADER]: Becoming Leader. State: Replica: 26bc508455ea47df86893d2e34a1d236, State: Running, Role: LEADER
I20260807 20:12:25.473101 26023 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bc508455ea47df86893d2e34a1d236" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35333 } }
I20260807 20:12:25.473495 26021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26bc508455ea47df86893d2e34a1d236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bc508455ea47df86893d2e34a1d236" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35333 } } }
I20260807 20:12:25.473594 26021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:25.473774 26024 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26bc508455ea47df86893d2e34a1d236. Latest consensus state: current_term: 1 leader_uuid: "26bc508455ea47df86893d2e34a1d236" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26bc508455ea47df86893d2e34a1d236" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 35333 } } }
I20260807 20:12:25.473871 26024 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:25.474022 26031 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:25.474766 26031 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:25.476411 26031 catalog_manager.cc:1347] Generated new cluster ID: 75ad5640899f463eabb23e6960430735
I20260807 20:12:25.476521 26031 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 20:12:25 dist-test-slave-1x4g krb5kdc[25933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:25.485001 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:25.472409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33241 (local address 127.3.15.190:35333)
0807 20:12:25.472622 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:25.472633 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:25.472668 (+ 35us) server_negotiation.cc:407] Connection header received
0807 20:12:25.472820 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:25.472828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:25.472892 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:25.474007 (+ 1115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:25.474141 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:25.475562 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:25.476681 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:25.477341 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:25.477815 (+ 474us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:25.480437 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:25.480442 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:25.480442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:25.480454 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:25.483020 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:25.483487 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:25.483491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:25.483491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:25.483626 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:25.483876 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:25.483877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:25.483878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:25.484218 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:25.484382 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:25.484498 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 20:12:25.484744 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":65,"threads_started":1}
I20260807 20:12:25.486384 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:25.487895 26031 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:25.488610 26031 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:25.493728 26031 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26bc508455ea47df86893d2e34a1d236: Generated new TSK 0
I20260807 20:12:25.493950 26031 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:25.494767 26022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:25.487922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49467 (local address 127.3.15.190:35333)
0807 20:12:25.488017 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:25.488019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:25.488286 (+ 267us) server_negotiation.cc:407] Connection header received
0807 20:12:25.488332 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:25.488332 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:25.488342 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:25.488397 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:25.489078 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:25.489601 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:25.490364 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:25.490606 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:25.490696 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:25.491918 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:25.491919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:25.491919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:25.491927 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:25.493512 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:25.493967 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:25.493968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:25.493969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:25.494031 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:25.494278 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:25.494279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:25.494280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:25.494404 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:25.494510 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:25.494611 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 20:12:25.494677 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 20:12:25.496470 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 25946
W20260807 20:12:25.502611 25931 connection.cc:535] client connection to 127.3.15.190:35333 recv error: Network error: failed to read from TLS socket (remote: 127.3.15.190:35333): Connection reset by peer (error 104)
2026-08-07T20:12:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:25 dist-test-slave-1x4g krb5kdc[26049](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:25 dist-test-slave-1x4g krb5kdc[26049](info): set up 2 sockets
Aug 07 20:12:25 dist-test-slave-1x4g krb5kdc[26049](info): commencing operation
krb5kdc: starting...
W20260807 20:12:26.561414 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:26 dist-test-slave-1x4g krb5kdc[26049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:26.687326 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42927
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:38443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:42927
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:26.771405 26062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:26.771624 26062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:26.771648 26062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:26.772843 26062 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:26.772887 26062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:26.772902 26062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:26.772914 26062 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:26.772974 26062 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:26.774458 26062 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:38443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:42927
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:26.774724 26062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:26.774958 26062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:26.780006 26068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:26.780088 26062 server_base.cc:1034] running on GCE node
W20260807 20:12:26.780027 26070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:26.779992 26067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:26.780454 26062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:26.780768 26062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:26.781994 26062 hybrid_clock.cc:648] HybridClock initialized: now 1786133546781977 us; error 28 us; skew 500 ppm
Aug 07 20:12:26 dist-test-slave-1x4g krb5kdc[26049](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133546, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:26.784505 26062 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:26.785013 26062 webserver.cc:460] Webserver started at http://127.3.15.190:38409/ using document root <none> and password file <none>
I20260807 20:12:26.785249 26062 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:26.785319 26062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:26.785473 26062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:26.786571 26062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "b82b1c23fa944479bdc1c3305983544d"
format_stamp: "Formatted at 2026-08-07 20:12:26 on dist-test-slave-1x4g"
server_key: "50bffc8338694c1126a7412de2967f1e"
server_key_iv: "bb1ebe032253dcca0c496b331b3a51e0"
server_key_version: "encryptionkey@0"
I20260807 20:12:26.786957 26062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "b82b1c23fa944479bdc1c3305983544d"
format_stamp: "Formatted at 2026-08-07 20:12:26 on dist-test-slave-1x4g"
server_key: "50bffc8338694c1126a7412de2967f1e"
server_key_iv: "bb1ebe032253dcca0c496b331b3a51e0"
server_key_version: "encryptionkey@0"
I20260807 20:12:26.788321 26062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:26.789158 26077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:26.789336 26062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 20:12:26.789450 26062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "b82b1c23fa944479bdc1c3305983544d"
format_stamp: "Formatted at 2026-08-07 20:12:26 on dist-test-slave-1x4g"
server_key: "50bffc8338694c1126a7412de2967f1e"
server_key_iv: "bb1ebe032253dcca0c496b331b3a51e0"
server_key_version: "encryptionkey@0"
I20260807 20:12:26.789528 26062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:26.800374 26062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:26.802227 26062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:26.802403 26062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:26.806489 26062 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:42927
I20260807 20:12:26.806520 26133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:42927 every 8 connection(s)
I20260807 20:12:26.806871 26062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:26.807801 26134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:26.810148 26134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d: Bootstrap starting.
I20260807 20:12:26.810698 26134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:26.810935 26134 log.cc:824] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:26.811612 26134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d: No bootstrap required, opened a new log
I20260807 20:12:26.812631 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26062
I20260807 20:12:26.812794 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:26.812997 3134 external_mini_cluster.cc:1428] Setting key 7a95d6a91243663b0c8d6b07c8bc5534
I20260807 20:12:26.813730 26134 raft_consensus.cc:348] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b1c23fa944479bdc1c3305983544d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42927 } }
I20260807 20:12:26.813908 26134 raft_consensus.cc:374] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:26.813946 26134 raft_consensus.cc:729] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b82b1c23fa944479bdc1c3305983544d, State: Initialized, Role: FOLLOWER
I20260807 20:12:26.814074 26134 consensus_queue.cc:260] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b1c23fa944479bdc1c3305983544d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42927 } }
I20260807 20:12:26.814172 26134 raft_consensus.cc:388] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:26.814205 26134 raft_consensus.cc:482] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:26.814301 26134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:26.815060 26134 raft_consensus.cc:504] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b1c23fa944479bdc1c3305983544d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42927 } }
I20260807 20:12:26.815176 26134 leader_election.cc:302] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b82b1c23fa944479bdc1c3305983544d; no voters:
I20260807 20:12:26.815335 26134 leader_election.cc:288] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:26.815392 26140 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:26.815567 26134 sys_catalog.cc:564] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:26.815598 26140 raft_consensus.cc:686] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [term 1 LEADER]: Becoming Leader. State: Replica: b82b1c23fa944479bdc1c3305983544d, State: Running, Role: LEADER
I20260807 20:12:26.815717 26140 consensus_queue.cc:237] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b1c23fa944479bdc1c3305983544d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42927 } }
I20260807 20:12:26.816164 26141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b82b1c23fa944479bdc1c3305983544d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b1c23fa944479bdc1c3305983544d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42927 } } }
I20260807 20:12:26.816143 26140 sys_catalog.cc:455] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b82b1c23fa944479bdc1c3305983544d. Latest consensus state: current_term: 1 leader_uuid: "b82b1c23fa944479bdc1c3305983544d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b82b1c23fa944479bdc1c3305983544d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42927 } } }
I20260807 20:12:26.816243 26141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:26.816248 26140 sys_catalog.cc:458] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:26.816821 26149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:26.817351 26149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:26.819314 26149 catalog_manager.cc:1347] Generated new cluster ID: d13476b7fac948e293576cb3eed80486
I20260807 20:12:26.819373 26149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:26.819677 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:26.813548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51029 (local address 127.3.15.190:42927)
0807 20:12:26.813764 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:26.813773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:26.813806 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:26.813918 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:26.813925 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:26.813982 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:26.814134 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:26.814419 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:26.816216 (+ 1797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:26.817470 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:26.818065 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:26.818177 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:26.818872 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:26.818877 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:26.818877 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:26.819129 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:26.819167 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:26.819182 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:26.819268 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:26.819431 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 20:12:26.819460 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":107,"threads_started":1}
I20260807 20:12:26.820603 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:26.825618 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:26.822005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55751 (local address 127.3.15.190:42927)
0807 20:12:26.822036 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:26.822038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:26.822149 (+ 111us) server_negotiation.cc:407] Connection header received
0807 20:12:26.822204 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:26.822205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:26.822214 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:26.822268 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:26.822394 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:26.823103 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:26.824207 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:26.824551 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:26.824621 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:26.825353 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:26.825354 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:26.825355 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:26.825359 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:26.825372 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:26.825376 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:26.825424 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:26.825533 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 20:12:26.825545 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":16}
I20260807 20:12:26.829476 26149 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:26.830585 26149 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:26.834388 26149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b82b1c23fa944479bdc1c3305983544d: Generated new TSK 0
I20260807 20:12:26.834550 26149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:26.837989 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26062
W20260807 20:12:26.844058 26047 connection.cc:535] client connection to 127.3.15.190:42927 recv error: Network error: failed to read from TLS socket (remote: 127.3.15.190:42927): Connection reset by peer (error 104)
2026-08-07T20:12:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:26 dist-test-slave-1x4g krb5kdc[26167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:26 dist-test-slave-1x4g krb5kdc[26167](info): set up 2 sockets
Aug 07 20:12:26 dist-test-slave-1x4g krb5kdc[26167](info): commencing operation
krb5kdc: starting...
W20260807 20:12:27.917579 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:28 dist-test-slave-1x4g krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:28.043419 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42327
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:37085
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:42327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:28.126827 26181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:28.127038 26181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:28.127060 26181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:28.128135 26181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:28.128172 26181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:28.128186 26181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:28.128198 26181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:28.128211 26181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:28.129701 26181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:37085
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:42327
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:42327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:28.129909 26181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:28.130084 26181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:28.134940 26189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:28.134966 26187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:28.135159 26181 server_base.cc:1034] running on GCE node
W20260807 20:12:28.135250 26186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:28.135490 26181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:28.135823 26181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:28.136993 26181 hybrid_clock.cc:648] HybridClock initialized: now 1786133548137000 us; error 68 us; skew 500 ppm
Aug 07 20:12:28 dist-test-slave-1x4g krb5kdc[26167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133548, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:28.139611 26181 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:28.139984 26181 webserver.cc:460] Webserver started at http://127.3.15.190:37091/ using document root <none> and password file <none>
I20260807 20:12:28.140244 26181 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:28.140288 26181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:28.140424 26181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:28.141538 26181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "39e0daa3e30b4722898251d5800931dd"
format_stamp: "Formatted at 2026-08-07 20:12:28 on dist-test-slave-1x4g"
server_key: "e526ecf70635e6d2be308b501492f1f6"
server_key_iv: "db00137c27f36e7a4857da7ce357422e"
server_key_version: "encryptionkey@0"
I20260807 20:12:28.141919 26181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "39e0daa3e30b4722898251d5800931dd"
format_stamp: "Formatted at 2026-08-07 20:12:28 on dist-test-slave-1x4g"
server_key: "e526ecf70635e6d2be308b501492f1f6"
server_key_iv: "db00137c27f36e7a4857da7ce357422e"
server_key_version: "encryptionkey@0"
I20260807 20:12:28.143308 26181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:28.144075 26197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:28.144264 26181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:28.144376 26181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "39e0daa3e30b4722898251d5800931dd"
format_stamp: "Formatted at 2026-08-07 20:12:28 on dist-test-slave-1x4g"
server_key: "e526ecf70635e6d2be308b501492f1f6"
server_key_iv: "db00137c27f36e7a4857da7ce357422e"
server_key_version: "encryptionkey@0"
I20260807 20:12:28.144452 26181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:28.147898 26181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:28.150297 26181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:28.150463 26181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:28.154713 26181 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:42327
I20260807 20:12:28.154747 26253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:42327 every 8 connection(s)
I20260807 20:12:28.155134 26181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:28.156169 26254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:28.158562 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26181
I20260807 20:12:28.158576 26254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd: Bootstrap starting.
I20260807 20:12:28.158706 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:28.158893 3134 external_mini_cluster.cc:1428] Setting key cf0cc6dd2c1fccf8941aa17a3eb8dbdc
I20260807 20:12:28.159296 26254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:28.159688 26254 log.cc:824] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:28.160517 26254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd: No bootstrap required, opened a new log
I20260807 20:12:28.161397 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:28.159395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42747 (local address 127.3.15.190:42327)
0807 20:12:28.159603 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:28.159611 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:28.159639 (+ 28us) server_negotiation.cc:407] Connection header received
0807 20:12:28.160120 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:28.160127 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:28.160164 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:28.160287 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:28.160951 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:28.160954 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:28.160954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:28.160958 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:28.160999 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:28.161010 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:28.161087 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:28.161151 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:12:28.161203 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":98,"threads_started":1}
I20260807 20:12:28.162343 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:28.163026 26254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0daa3e30b4722898251d5800931dd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42327 } }
I20260807 20:12:28.163210 26254 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:28.163283 26254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e0daa3e30b4722898251d5800931dd, State: Initialized, Role: FOLLOWER
I20260807 20:12:28.163439 26254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0daa3e30b4722898251d5800931dd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42327 } }
I20260807 20:12:28.163627 26254 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:28.163686 26254 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:28.163739 26254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:28.164252 26254 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0daa3e30b4722898251d5800931dd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42327 } }
I20260807 20:12:28.164373 26254 leader_election.cc:302] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39e0daa3e30b4722898251d5800931dd; no voters:
I20260807 20:12:28.164556 26254 leader_election.cc:288] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:28.164609 26263 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:28.164788 26263 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [term 1 LEADER]: Becoming Leader. State: Replica: 39e0daa3e30b4722898251d5800931dd, State: Running, Role: LEADER
I20260807 20:12:28.164845 26254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:28.164880 26263 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0daa3e30b4722898251d5800931dd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42327 } }
I20260807 20:12:28.165247 26265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39e0daa3e30b4722898251d5800931dd. Latest consensus state: current_term: 1 leader_uuid: "39e0daa3e30b4722898251d5800931dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0daa3e30b4722898251d5800931dd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42327 } } }
I20260807 20:12:28.165535 26265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:28.165447 26264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39e0daa3e30b4722898251d5800931dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0daa3e30b4722898251d5800931dd" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 42327 } } }
I20260807 20:12:28.165808 26264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:28.165901 26272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:28.166460 26272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:28.167946 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:28.166422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34677 (local address 127.3.15.190:42327)
0807 20:12:28.166461 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:28.166463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:28.166650 (+ 187us) server_negotiation.cc:407] Connection header received
0807 20:12:28.166738 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:28.166739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:28.166745 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:28.166796 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:28.167698 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:28.167699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:28.167700 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:28.167703 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:28.167717 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:28.167721 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:28.167781 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:28.167844 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 20:12:28.167858 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:28.168160 26272 catalog_manager.cc:1347] Generated new cluster ID: 7a6fd8addf904081b92ef8b8a7e6fcb6
I20260807 20:12:28.168215 26272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:28.191241 26272 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:28.191427 26272 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:28.194348 26272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39e0daa3e30b4722898251d5800931dd: Generated new TSK 0
I20260807 20:12:28.194522 26272 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:28.198374 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26181
2026-08-07T20:12:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1360 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:28 dist-test-slave-1x4g krb5kdc[26286](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:28 dist-test-slave-1x4g krb5kdc[26286](info): set up 2 sockets
Aug 07 20:12:28 dist-test-slave-1x4g krb5kdc[26286](info): commencing operation
krb5kdc: starting...
W20260807 20:12:29.337934 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:29 dist-test-slave-1x4g krb5kdc[26286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:29.467944 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34481
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:44455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:34481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:29.550773 26299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:29.551010 26299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:29.551046 26299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:29.552201 26299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:29.552270 26299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:29.552295 26299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:29.552316 26299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:29.552347 26299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:29.553843 26299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:44455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:34481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:34481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:29.554123 26299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:29.554352 26299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:29.559216 26304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:29.559263 26305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:29.559228 26307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:29.559319 26299 server_base.cc:1034] running on GCE node
I20260807 20:12:29.559695 26299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:29.559970 26299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:29.561112 26299 hybrid_clock.cc:648] HybridClock initialized: now 1786133549561094 us; error 39 us; skew 500 ppm
Aug 07 20:12:29 dist-test-slave-1x4g krb5kdc[26286](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133549, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:29.563684 26299 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:29.564143 26299 webserver.cc:460] Webserver started at http://127.3.15.190:38551/ using document root <none> and password file <none>
I20260807 20:12:29.564381 26299 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:29.564448 26299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:29.564603 26299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:29.565688 26299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "2fd87501066e4f069eb1c0f001ba39f5"
format_stamp: "Formatted at 2026-08-07 20:12:29 on dist-test-slave-1x4g"
server_key: "62051c64ba2206961620c2c12c736b67"
server_key_iv: "0ff3c9196db0bf7a937045f4d923f21e"
server_key_version: "encryptionkey@0"
I20260807 20:12:29.566063 26299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "2fd87501066e4f069eb1c0f001ba39f5"
format_stamp: "Formatted at 2026-08-07 20:12:29 on dist-test-slave-1x4g"
server_key: "62051c64ba2206961620c2c12c736b67"
server_key_iv: "0ff3c9196db0bf7a937045f4d923f21e"
server_key_version: "encryptionkey@0"
I20260807 20:12:29.567366 26299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 20:12:29.568080 26314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:29.568257 26299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:29.568364 26299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "2fd87501066e4f069eb1c0f001ba39f5"
format_stamp: "Formatted at 2026-08-07 20:12:29 on dist-test-slave-1x4g"
server_key: "62051c64ba2206961620c2c12c736b67"
server_key_iv: "0ff3c9196db0bf7a937045f4d923f21e"
server_key_version: "encryptionkey@0"
I20260807 20:12:29.568439 26299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:29.585073 26299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:29.587420 26299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:29.587595 26299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:29.591856 26299 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:34481
I20260807 20:12:29.591918 26367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:34481 every 8 connection(s)
I20260807 20:12:29.592252 26299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:29.593286 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26299
I20260807 20:12:29.593299 26368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:29.593377 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:29.593586 3134 external_mini_cluster.cc:1428] Setting key 482f364e90082cbc3c0ae8eb0659414d
I20260807 20:12:29.596680 26368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5: Bootstrap starting.
I20260807 20:12:29.597393 26368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:29.597682 26368 log.cc:824] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:29.598433 26368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5: No bootstrap required, opened a new log
Aug 07 20:12:29 dist-test-slave-1x4g krb5kdc[26286](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:29.601262 26368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd87501066e4f069eb1c0f001ba39f5" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34481 } }
I20260807 20:12:29.601404 26368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:29.601431 26368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd87501066e4f069eb1c0f001ba39f5, State: Initialized, Role: FOLLOWER
I20260807 20:12:29.601589 26368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd87501066e4f069eb1c0f001ba39f5" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34481 } }
I20260807 20:12:29.601681 26368 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:29.601729 26368 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:29.601837 26368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:29.602658 26368 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd87501066e4f069eb1c0f001ba39f5" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34481 } }
I20260807 20:12:29.602784 26368 leader_election.cc:302] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fd87501066e4f069eb1c0f001ba39f5; no voters:
I20260807 20:12:29.602975 26368 leader_election.cc:288] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:29.603042 26373 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:29.603210 26373 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [term 1 LEADER]: Becoming Leader. State: Replica: 2fd87501066e4f069eb1c0f001ba39f5, State: Running, Role: LEADER
I20260807 20:12:29.603250 26368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:29.603353 26373 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd87501066e4f069eb1c0f001ba39f5" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34481 } }
I20260807 20:12:29.603776 26374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fd87501066e4f069eb1c0f001ba39f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd87501066e4f069eb1c0f001ba39f5" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34481 } } }
I20260807 20:12:29.603865 26374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:29.603783 26375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fd87501066e4f069eb1c0f001ba39f5. Latest consensus state: current_term: 1 leader_uuid: "2fd87501066e4f069eb1c0f001ba39f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fd87501066e4f069eb1c0f001ba39f5" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 34481 } } }
I20260807 20:12:29.604166 26375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:29.604362 26380 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:29.604946 26380 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:29.605808 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:29.594114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42211 (local address 127.3.15.190:34481)
0807 20:12:29.594321 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:29.594331 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:29.594364 (+ 33us) server_negotiation.cc:407] Connection header received
0807 20:12:29.594907 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:29.594917 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:29.594994 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:29.595151 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:29.595729 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:29.597134 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:29.598102 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:29.598564 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:29.598650 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:29.601337 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:29.601342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:29.601342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:29.601352 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:29.603448 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:29.603924 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:29.603928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:29.603928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:29.604055 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:29.604313 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:29.604314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:29.604314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:29.604673 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:29.605382 (+ 709us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:29.605465 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:12:29.605597 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260807 20:12:29.606806 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:29.609275 26380 catalog_manager.cc:1347] Generated new cluster ID: 1c89525e68b6475c9f3c8bc161c6f1ca
I20260807 20:12:29.609328 26380 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:29.617642 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:29.608240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53715 (local address 127.3.15.190:34481)
0807 20:12:29.608281 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:29.608283 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:29.608386 (+ 103us) server_negotiation.cc:407] Connection header received
0807 20:12:29.608453 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:29.608454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:29.608462 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:29.608507 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:29.609186 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:29.609701 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:29.613588 (+ 3887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:29.613811 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:29.613886 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:29.615050 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:29.615051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:29.615052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:29.615065 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:29.616472 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:29.616876 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:29.616878 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:29.616878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:29.616938 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:29.617148 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:29.617149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:29.617149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:29.617258 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:29.617358 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:29.617467 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 20:12:29.617538 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:29.638203 26380 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:29.638746 26380 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:29.646502 26380 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fd87501066e4f069eb1c0f001ba39f5: Generated new TSK 0
I20260807 20:12:29.646719 26380 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:29.653466 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26299
2026-08-07T20:12:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:29 dist-test-slave-1x4g krb5kdc[26401](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:29 dist-test-slave-1x4g krb5kdc[26401](info): set up 2 sockets
Aug 07 20:12:29 dist-test-slave-1x4g krb5kdc[26401](info): commencing operation
krb5kdc: starting...
W20260807 20:12:30.722486 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:30 dist-test-slave-1x4g krb5kdc[26401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:30.851431 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33025
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:42931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:30.936178 26416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:30.936393 26416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:30.936416 26416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:30.937709 26416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:30.937754 26416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:30.937769 26416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:30.937783 26416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:30.937794 26416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:30.939286 26416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:42931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:30.939538 26416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:30.939774 26416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:30.944607 26424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:30.944854 26416 server_base.cc:1034] running on GCE node
W20260807 20:12:30.944615 26421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:30.944581 26422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:30.945228 26416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:30.945515 26416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:30.946683 26416 hybrid_clock.cc:648] HybridClock initialized: now 1786133550946677 us; error 47 us; skew 500 ppm
Aug 07 20:12:30 dist-test-slave-1x4g krb5kdc[26401](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133550, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:30.953284 26416 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:30.953694 26416 webserver.cc:460] Webserver started at http://127.3.15.190:46835/ using document root <none> and password file <none>
I20260807 20:12:30.953879 26416 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:30.953919 26416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:30.954013 26416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:30.955045 26416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "0976cebc240647e78fb005f317e4a116"
format_stamp: "Formatted at 2026-08-07 20:12:30 on dist-test-slave-1x4g"
server_key: "710344b6abd85929085c6166f3efa3ed"
server_key_iv: "69eb3c659ff9b2d095b10361e95ff1ed"
server_key_version: "encryptionkey@0"
I20260807 20:12:30.955395 26416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "0976cebc240647e78fb005f317e4a116"
format_stamp: "Formatted at 2026-08-07 20:12:30 on dist-test-slave-1x4g"
server_key: "710344b6abd85929085c6166f3efa3ed"
server_key_iv: "69eb3c659ff9b2d095b10361e95ff1ed"
server_key_version: "encryptionkey@0"
I20260807 20:12:30.956707 26416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:12:30.957562 26431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:30.957788 26416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:30.957883 26416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "0976cebc240647e78fb005f317e4a116"
format_stamp: "Formatted at 2026-08-07 20:12:30 on dist-test-slave-1x4g"
server_key: "710344b6abd85929085c6166f3efa3ed"
server_key_iv: "69eb3c659ff9b2d095b10361e95ff1ed"
server_key_version: "encryptionkey@0"
I20260807 20:12:30.957943 26416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:30.971499 26416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:30.973894 26416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:30.974006 26416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:30.978102 26416 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:33025
I20260807 20:12:30.978184 26484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:33025 every 8 connection(s)
I20260807 20:12:30.978535 26416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:30.979463 26485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:30.981884 26485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116: Bootstrap starting.
I20260807 20:12:30.982537 26485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:30.982849 26485 log.cc:824] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:30.983642 26485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116: No bootstrap required, opened a new log
I20260807 20:12:30.985314 26485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0976cebc240647e78fb005f317e4a116" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33025 } }
I20260807 20:12:30.985482 26485 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:30.985534 26485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0976cebc240647e78fb005f317e4a116, State: Initialized, Role: FOLLOWER
I20260807 20:12:30.985689 26485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0976cebc240647e78fb005f317e4a116" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33025 } }
I20260807 20:12:30.985774 26485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:30.985819 26485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:30.985872 26485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:30.986752 26485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0976cebc240647e78fb005f317e4a116" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33025 } }
I20260807 20:12:30.986877 26485 leader_election.cc:302] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0976cebc240647e78fb005f317e4a116; no voters:
I20260807 20:12:30.987056 26485 leader_election.cc:288] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:30.987090 26488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:30.987280 26488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [term 1 LEADER]: Becoming Leader. State: Replica: 0976cebc240647e78fb005f317e4a116, State: Running, Role: LEADER
I20260807 20:12:30.987350 26485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:30.987396 26488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0976cebc240647e78fb005f317e4a116" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33025 } }
I20260807 20:12:30.987443 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26416
I20260807 20:12:30.987594 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:30.987797 3134 external_mini_cluster.cc:1428] Setting key 5b296e9c81f2730322764b4cd9c589c7
I20260807 20:12:30.987847 26489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0976cebc240647e78fb005f317e4a116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0976cebc240647e78fb005f317e4a116" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33025 } } }
I20260807 20:12:30.987943 26489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:30.988153 26490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0976cebc240647e78fb005f317e4a116. Latest consensus state: current_term: 1 leader_uuid: "0976cebc240647e78fb005f317e4a116" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0976cebc240647e78fb005f317e4a116" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33025 } } }
I20260807 20:12:30.988288 26490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:30.988453 26497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:30.989255 26497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:30.990936 26497 catalog_manager.cc:1347] Generated new cluster ID: dd73a958a87b428ca46e7423008d04d0
I20260807 20:12:30.990988 26497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:30.994717 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:30.988277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58773 (local address 127.3.15.190:33025)
0807 20:12:30.989093 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:30.989103 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:30.989159 (+ 56us) server_negotiation.cc:407] Connection header received
0807 20:12:30.989317 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:30.989324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:30.989380 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:30.989792 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:30.989886 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:30.991246 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:30.992358 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:30.993037 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:30.993154 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:30.993897 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:30.993901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:30.993901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:30.994126 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:30.994167 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:30.994183 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:30.994255 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:30.994319 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:12:30.994494 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":714,"thread_start_us":87,"threads_started":1}
I20260807 20:12:30.995654 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:31.000730 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:30.997304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43797 (local address 127.3.15.190:33025)
0807 20:12:30.997348 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:30.997350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:30.997491 (+ 141us) server_negotiation.cc:407] Connection header received
0807 20:12:30.997551 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:30.997552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:30.997562 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:30.997611 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:30.997745 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:30.998434 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:30.999449 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:30.999733 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:30.999795 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:31.000493 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:31.000494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:31.000494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:31.000500 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:31.000515 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:31.000520 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:31.000570 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:31.000634 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 20:12:31.000658 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 20:12:31.002553 26497 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:31.003242 26497 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:31.008132 26497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0976cebc240647e78fb005f317e4a116: Generated new TSK 0
I20260807 20:12:31.008315 26497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:31.010694 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26416
2026-08-07T20:12:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:31 dist-test-slave-1x4g krb5kdc[26516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:31 dist-test-slave-1x4g krb5kdc[26516](info): set up 2 sockets
Aug 07 20:12:31 dist-test-slave-1x4g krb5kdc[26516](info): commencing operation
krb5kdc: starting...
W20260807 20:12:32.122334 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:32 dist-test-slave-1x4g krb5kdc[26516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:32.249850 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45263
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:41597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:45263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:32.331470 26529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:32.331669 26529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:32.331693 26529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:32.332819 26529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:32.332856 26529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:32.332891 26529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:32.332904 26529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:32.332916 26529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:32.334367 26529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:41597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:45263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:45263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:32.334621 26529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:32.334852 26529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:32.339695 26537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:32.339716 26535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:32.339723 26534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:32.339747 26529 server_base.cc:1034] running on GCE node
I20260807 20:12:32.340147 26529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:32.340471 26529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:32.341611 26529 hybrid_clock.cc:648] HybridClock initialized: now 1786133552341591 us; error 42 us; skew 500 ppm
Aug 07 20:12:32 dist-test-slave-1x4g krb5kdc[26516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133552, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:32.344149 26529 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:32.344620 26529 webserver.cc:460] Webserver started at http://127.3.15.190:44989/ using document root <none> and password file <none>
I20260807 20:12:32.344950 26529 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:32.345036 26529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:32.345172 26529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:32.346254 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "0b9a6708d8654339a3081bba810bb5b9"
format_stamp: "Formatted at 2026-08-07 20:12:32 on dist-test-slave-1x4g"
server_key: "2d314712aa84366bbc20043ba144e295"
server_key_iv: "ecf231b5f91b356b06c0b21a932e0156"
server_key_version: "encryptionkey@0"
I20260807 20:12:32.346618 26529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "0b9a6708d8654339a3081bba810bb5b9"
format_stamp: "Formatted at 2026-08-07 20:12:32 on dist-test-slave-1x4g"
server_key: "2d314712aa84366bbc20043ba144e295"
server_key_iv: "ecf231b5f91b356b06c0b21a932e0156"
server_key_version: "encryptionkey@0"
I20260807 20:12:32.347945 26529 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:32.348663 26545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:32.348835 26529 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:32.348928 26529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "0b9a6708d8654339a3081bba810bb5b9"
format_stamp: "Formatted at 2026-08-07 20:12:32 on dist-test-slave-1x4g"
server_key: "2d314712aa84366bbc20043ba144e295"
server_key_iv: "ecf231b5f91b356b06c0b21a932e0156"
server_key_version: "encryptionkey@0"
I20260807 20:12:32.348984 26529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:32.352149 26529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:32.354421 26529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:32.354525 26529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:32.358470 26529 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:45263
I20260807 20:12:32.358518 26600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:45263 every 8 connection(s)
I20260807 20:12:32.358816 26529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:32.359786 26601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:32.362022 26601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9: Bootstrap starting.
I20260807 20:12:32.362588 26601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:32.362874 26601 log.cc:824] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:32.363545 26601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9: No bootstrap required, opened a new log
I20260807 20:12:32.364986 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26529
I20260807 20:12:32.365082 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:32.365260 3134 external_mini_cluster.cc:1428] Setting key 071b6d3880ae1c41960a2e118b6ec8bf
I20260807 20:12:32.365273 26601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9a6708d8654339a3081bba810bb5b9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45263 } }
I20260807 20:12:32.365489 26601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 20:12:32.365533 26601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b9a6708d8654339a3081bba810bb5b9, State: Initialized, Role: FOLLOWER
I20260807 20:12:32.365684 26601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9a6708d8654339a3081bba810bb5b9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45263 } }
I20260807 20:12:32.365780 26601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 20:12:32.365823 26601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 20:12:32.365880 26601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:32.366986 26601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9a6708d8654339a3081bba810bb5b9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45263 } }
I20260807 20:12:32.367125 26601 leader_election.cc:302] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b9a6708d8654339a3081bba810bb5b9; no voters:
I20260807 20:12:32.367314 26601 leader_election.cc:288] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 20:12:32.367370 26607 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:32.367506 26607 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [term 1 LEADER]: Becoming Leader. State: Replica: 0b9a6708d8654339a3081bba810bb5b9, State: Running, Role: LEADER
I20260807 20:12:32.367547 26606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:32.365759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54105 (local address 127.3.15.190:45263)
0807 20:12:32.365980 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:32.365988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:32.366025 (+ 37us) server_negotiation.cc:407] Connection header received
0807 20:12:32.366131 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:32.366136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:32.366158 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:32.366267 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:32.366920 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:32.366923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:32.366923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:32.366927 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:32.366968 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:32.366979 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:32.367035 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:32.367200 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 20:12:32.367248 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":93,"threads_started":1}
I20260807 20:12:32.367583 26601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:32.367599 26607 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9a6708d8654339a3081bba810bb5b9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45263 } }
I20260807 20:12:32.367967 26609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b9a6708d8654339a3081bba810bb5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9a6708d8654339a3081bba810bb5b9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45263 } } }
I20260807 20:12:32.368079 26609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:32.367957 26607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b9a6708d8654339a3081bba810bb5b9. Latest consensus state: current_term: 1 leader_uuid: "0b9a6708d8654339a3081bba810bb5b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b9a6708d8654339a3081bba810bb5b9" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 45263 } } }
I20260807 20:12:32.368278 26607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:32.368404 26614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:32.368438 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:32.368955 26614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:32.370874 26614 catalog_manager.cc:1347] Generated new cluster ID: 9ecf6fb3e68747b9bd2b153feb9dd68e
I20260807 20:12:32.370935 26614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:32.371197 26606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:32.370036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43607 (local address 127.3.15.190:45263)
0807 20:12:32.370086 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:32.370088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:32.370323 (+ 235us) server_negotiation.cc:407] Connection header received
0807 20:12:32.370370 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:32.370371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:32.370376 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:32.370435 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:32.370945 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:32.370947 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:32.370947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 20:12:32.370952 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:32.370965 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 20:12:32.370970 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:32.371020 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:32.371103 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 20:12:32.371115 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:12:32.377228 26614 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:32.377383 26614 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:32.386852 26614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b9a6708d8654339a3081bba810bb5b9: Generated new TSK 0
I20260807 20:12:32.387084 26614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:32.390218 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26529
2026-08-07T20:12:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1379 ms)
[----------] 18 tests from AuthTokenIssuingTest (24830 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:32 dist-test-slave-1x4g krb5kdc[26634](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:32 dist-test-slave-1x4g krb5kdc[26634](info): set up 2 sockets
Aug 07 20:12:32 dist-test-slave-1x4g krb5kdc[26634](info): commencing operation
krb5kdc: starting...
W20260807 20:12:33.469529 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:33.597934 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33293
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33293,127.3.15.189:35827,127.3.15.188:44363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:33.682191 26647 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:12:33.682390 26647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:33.682408 26647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:33.682428 26647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:33.683471 26647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:33.683504 26647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:33.683519 26647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:33.683531 26647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:33.683543 26647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:33.684945 26647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33293,127.3.15.189:35827,127.3.15.188:44363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:33293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:33.685132 26647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:33.685289 26647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:33.690050 26653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:33.690061 26655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:33.690160 26652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:33.690089 26647 server_base.cc:1034] running on GCE node
I20260807 20:12:33.690500 26647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:33.690783 26647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:33.691953 26647 hybrid_clock.cc:648] HybridClock initialized: now 1786133553691935 us; error 37 us; skew 500 ppm
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:33.694443 26647 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:33.694811 26647 webserver.cc:460] Webserver started at http://127.3.15.190:32853/ using document root <none> and password file <none>
I20260807 20:12:33.694985 26647 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:33.695025 26647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:33.695122 26647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:33.696080 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "68df459d6d284e0abebdc3de0666b4d7"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "0a588c84f7a767933edae0138a03fcd1"
server_key_iv: "99f4a97f69083f88ca39641a663b771d"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.696396 26647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "68df459d6d284e0abebdc3de0666b4d7"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "0a588c84f7a767933edae0138a03fcd1"
server_key_iv: "99f4a97f69083f88ca39641a663b771d"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.697745 26647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 20:12:33.698422 26662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:33.698621 26647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260807 20:12:33.698729 26647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "68df459d6d284e0abebdc3de0666b4d7"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "0a588c84f7a767933edae0138a03fcd1"
server_key_iv: "99f4a97f69083f88ca39641a663b771d"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.698804 26647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:33.708397 26647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:33.710171 26647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:33.710331 26647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:33.714303 26647 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:33293
I20260807 20:12:33.714361 26717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:33293 every 8 connection(s)
I20260807 20:12:33.714721 26647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:33.715714 26718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:33.717337 26718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:33.722529 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26647
I20260807 20:12:33.722687 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:33.722857 3134 external_mini_cluster.cc:1428] Setting key 2072a6aedd8d4db914f0ca39a029d6fb
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:33.726778 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.718049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33293 (local address 127.0.0.1:33580)
0807 20:12:33.718202 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.718235 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:33.718702 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:33.718951 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:33.718953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:33.719192 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:33.719340 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.719344 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.720305 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.720306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.721051 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.721054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.721520 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.721521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.721589 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.722011 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:33.722018 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:33.723466 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:33.725350 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.725357 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.725359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.725553 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.725826 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.725827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.725827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.725913 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.726355 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:33.726359 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:33.726413 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 20:12:33.726483 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 20:12:33.726622 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":65,"threads_started":1}
I20260807 20:12:33.726892 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.718197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33580 (local address 127.3.15.190:33293)
0807 20:12:33.718321 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:33.718328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:33.718360 (+ 32us) server_negotiation.cc:407] Connection header received
0807 20:12:33.718832 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:33.718834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:33.718850 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:33.718924 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:33.719420 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.720227 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.721129 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.721431 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.721508 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.723548 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:33.723557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:33.723557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:33.723565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:33.725248 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.725657 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.725659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.725660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.725743 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.725982 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.725983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.725983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.726282 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:33.726368 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:33.726548 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 20:12:33.726712 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":93,"thread_start_us":66,"threads_started":1}
I20260807 20:12:33.727641 26718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:33.727977 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.727865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:32960)
0807 20:12:33.727909 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.727958 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:12:33.728152 26666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
I20260807 20:12:33.728612 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.728513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:32968)
0807 20:12:33.728571 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.728596 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:12:33.728744 26718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:35827: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.15.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab.
Entry for principal kudu/127.3.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab.
I20260807 20:12:33.751421 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:35827
--webserver_interface=127.3.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab
--principal=kudu/127.3.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33293,127.3.15.189:35827,127.3.15.188:44363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:12:33.777989 26718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } attempt: 1
I20260807 20:12:33.778465 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.778328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:32970)
0807 20:12:33.778394 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.778434 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 20:12:33.778861 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.778763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:32980)
0807 20:12:33.778811 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.778837 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:12:33.778970 26718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:35827: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
W20260807 20:12:33.832947 26725 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:12:33.833199 26725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:33.833230 26725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:33.833274 26725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:33.834385 26725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:33.834455 26725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:33.834481 26725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:33.834503 26725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:33.834532 26725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:33.836016 26725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33293,127.3.15.189:35827,127.3.15.188:44363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab
--principal=kudu/127.3.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:35827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:33.836309 26725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:33.836541 26725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 20:12:33.841477 26725 server_base.cc:1034] running on GCE node
W20260807 20:12:33.841462 26730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:33.841452 26731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:33.841462 26733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:33.841913 26725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:33.842216 26725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:33.843381 26725 hybrid_clock.cc:648] HybridClock initialized: now 1786133553843359 us; error 54 us; skew 500 ppm
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:33.845973 26725 init.cc:377] Logged in from keytab as kudu/127.3.15.189@KRBTEST.COM (short username kudu)
I20260807 20:12:33.846408 26725 webserver.cc:460] Webserver started at http://127.3.15.189:34999/ using document root <none> and password file <none>
I20260807 20:12:33.846637 26725 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:33.846706 26725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:33.846861 26725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:33.847127 26718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } attempt: 2
I20260807 20:12:33.847525 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.847378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:32990)
0807 20:12:33.847448 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.847487 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 20:12:33.847899 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.847799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:32998)
0807 20:12:33.847847 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.847875 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:12:33.848111 26718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:35827: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35827: connect: Connection refused (error 111)
I20260807 20:12:33.848351 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data/instance:
uuid: "8078f38c95024f03b9d9386802d6d31f"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "35d94c52d7a09234b3f542996ed5d2a3"
server_key_iv: "1e330a247816e07b3628c9b29b32e907"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.848730 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal/instance:
uuid: "8078f38c95024f03b9d9386802d6d31f"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "35d94c52d7a09234b3f542996ed5d2a3"
server_key_iv: "1e330a247816e07b3628c9b29b32e907"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.850109 26725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:33.850826 26740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:33.850986 26725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:33.851109 26725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal
uuid: "8078f38c95024f03b9d9386802d6d31f"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "35d94c52d7a09234b3f542996ed5d2a3"
server_key_iv: "1e330a247816e07b3628c9b29b32e907"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.851184 26725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:33.857931 26725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:33.860118 26725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:33.860272 26725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:33.864140 26725 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.189:35827
I20260807 20:12:33.864174 26795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.189:35827 every 8 connection(s)
I20260807 20:12:33.864527 26725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
I20260807 20:12:33.865532 26796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:33.866178 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26725
I20260807 20:12:33.866286 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-1/wal/instance
I20260807 20:12:33.866456 3134 external_mini_cluster.cc:1428] Setting key 1ff36678fd8ab81e99df68b344fff889
I20260807 20:12:33.867216 26796 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.15.188@KRBTEST.COM; defaulting to no policy
I20260807 20:12:33.876734 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.867969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33590 (local address 127.3.15.190:33293)
0807 20:12:33.868025 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:33.868029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:33.868254 (+ 225us) server_negotiation.cc:407] Connection header received
0807 20:12:33.868915 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:33.868917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:33.868936 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:33.869008 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:33.869613 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.870378 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.871346 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.871681 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.871760 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.873894 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:33.873895 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:33.873896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:33.873905 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:33.875508 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.875879 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.875880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.875880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.875934 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.876163 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.876164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.876164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.876268 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:33.876319 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:33.876578 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 20:12:33.876672 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:12:33.876987 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.867943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33293 (local address 127.0.0.1:33590)
0807 20:12:33.868134 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.868168 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:33.868714 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:33.869077 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:33.869080 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:33.869325 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:33.869512 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.869516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.870472 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.870473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.871259 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.871260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.871779 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.871780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.871883 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.872399 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:33.872408 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:33.873798 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:33.875587 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.875595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.875597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.875807 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.875996 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.875996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.875997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.876105 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.876323 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:33.876328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:33.876439 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 20:12:33.876519 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:12:33.876690 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260807 20:12:33.877537 26796 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.188@KRBTEST.COM" created.
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:12:33.885251 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.877797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:33002)
0807 20:12:33.877831 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.877842 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:33.877876 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:33.878181 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:33.878182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:33.878321 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:33.878363 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.878364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.879447 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.879448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.879947 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.879949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.880436 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.880437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.880500 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.880972 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:33.880976 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:33.882432 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:33.884142 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.884143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.884143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.884340 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.884584 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.884585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.884585 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.884649 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.885029 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:33.885029 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:33.885066 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 20:12:33.885117 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 20:12:33.885188 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 20:12:33.885404 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.877876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33002 (local address 127.3.15.189:35827)
0807 20:12:33.878032 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:33.878036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:33.878052 (+ 16us) server_negotiation.cc:407] Connection header received
0807 20:12:33.878085 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:33.878087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:33.878097 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:33.878180 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:33.878444 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.879341 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.880028 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.880345 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.880436 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.882520 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:33.882527 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:33.882528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:33.882534 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:33.884055 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.884436 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.884440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.884441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.884511 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.884714 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.884715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.884716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.884947 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:33.885039 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:33.885159 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 20:12:33.885256 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260807 20:12:33.886163 26796 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:33.886693 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.886559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44936)
0807 20:12:33.886624 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.886674 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 20:12:33.886826 26744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
I20260807 20:12:33.887254 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.887159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44942)
0807 20:12:33.887203 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.887227 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:12:33.887377 26796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:44363: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab.
Entry for principal kudu/127.3.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab.
I20260807 20:12:33.896096 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:44363
--webserver_interface=127.3.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab
--principal=kudu/127.3.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:38407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:33293,127.3.15.189:35827,127.3.15.188:44363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:12:33.936720 26796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } attempt: 1
I20260807 20:12:33.937338 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.937197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44958)
0807 20:12:33.937263 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.937310 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 20:12:33.937729 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.937621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44970)
0807 20:12:33.937676 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.937709 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 20:12:33.937839 26796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:44363: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
I20260807 20:12:33.939246 26718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } attempt: 3
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:12:33.946326 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.939540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:33018)
0807 20:12:33.939584 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.939595 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:33.939652 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:33.939799 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:33.939800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:33.939973 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:33.940041 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.940043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.940752 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.940753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.941345 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:33.941348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.941769 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.941770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:33.941856 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.942322 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:33.942333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:33.943539 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:33.945261 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.945262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.945263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.945439 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.945659 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:33.945660 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:33.945660 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:33.945727 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:33.945973 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:33.945974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:33.946022 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 20:12:33.946123 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 20:12:33.946228 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260807 20:12:33.946326 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.939585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33018 (local address 127.3.15.189:35827)
0807 20:12:33.939643 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:33.939645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:33.939657 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:33.939722 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:33.939723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:33.939734 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:33.939791 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:33.940152 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.940659 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.941435 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:33.941678 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:33.941766 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:33.943630 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:33.943632 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:33.943633 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:33.943640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:33.945140 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.945513 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.945515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.945515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.945578 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:33.945787 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:33.945788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:33.945789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:33.945903 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:33.945981 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:33.946152 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 20:12:33.946228 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:12:33.946710 26718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:33.947088 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.946985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44972)
0807 20:12:33.947037 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.947068 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:12:33.947356 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.947294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44974)
0807 20:12:33.947320 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.947344 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260807 20:12:33.947418 26718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:44363: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
I20260807 20:12:33.978623 26718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } attempt: 1
I20260807 20:12:33.979127 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.978966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44984)
0807 20:12:33.979039 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.979079 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 20:12:33.979532 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:33.979440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44988)
0807 20:12:33.979489 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:33.979515 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 20:12:33.979651 26718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:44363: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
W20260807 20:12:33.981936 26803 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:12:33.982131 26803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:33.982147 26803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:33.982167 26803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:33.983227 26803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:33.983261 26803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:33.983275 26803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:33.983287 26803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:33.983299 26803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:33.984700 26803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:38407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:33293,127.3.15.189:35827,127.3.15.188:44363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab
--principal=kudu/127.3.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:33.984947 26803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:33.985142 26803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:33.989892 26809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:33.989884 26808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:33.989880 26811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:33.990134 26803 server_base.cc:1034] running on GCE node
I20260807 20:12:33.990391 26803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:33.990691 26803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:33.991860 26803 hybrid_clock.cc:648] HybridClock initialized: now 1786133553991847 us; error 28 us; skew 500 ppm
Aug 07 20:12:33 dist-test-slave-1x4g krb5kdc[26634](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:33.994366 26803 init.cc:377] Logged in from keytab as kudu/127.3.15.188@KRBTEST.COM (short username kudu)
I20260807 20:12:33.994868 26803 webserver.cc:460] Webserver started at http://127.3.15.188:39927/ using document root <none> and password file <none>
I20260807 20:12:33.995118 26803 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:33.995188 26803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:33.995342 26803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:33.996407 26803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data/instance:
uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "733a5c70d63a9d6d97fcc17783d385a4"
server_key_iv: "a0082f11f42481fef2cf639bf870a561"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.996863 26803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal/instance:
uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "733a5c70d63a9d6d97fcc17783d385a4"
server_key_iv: "a0082f11f42481fef2cf639bf870a561"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.998241 26803 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:33.998952 26818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:33.999130 26803 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:33.999230 26803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal
uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6"
format_stamp: "Formatted at 2026-08-07 20:12:33 on dist-test-slave-1x4g"
server_key: "733a5c70d63a9d6d97fcc17783d385a4"
server_key_iv: "a0082f11f42481fef2cf639bf870a561"
server_key_version: "encryptionkey@0"
I20260807 20:12:33.999310 26803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:34.006009 26796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } attempt: 2
I20260807 20:12:34.006443 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.006316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:44998)
0807 20:12:34.006376 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.006413 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:12:34.006816 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.006731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:45004)
0807 20:12:34.006774 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.006801 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:12:34.006922 26796 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:44363: Network error: Client connection negotiation failed: client connection to 127.3.15.188:44363: connect: Connection refused (error 111)
I20260807 20:12:34.010974 26803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:34.012755 26803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:34.012929 26803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:34.016768 26803 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.188:44363
I20260807 20:12:34.016857 26873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.188:44363 every 8 connection(s)
I20260807 20:12:34.017254 26803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
I20260807 20:12:34.018311 26874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:34.019872 26874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:34.021811 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 26803
I20260807 20:12:34.021976 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786133281907902-3134-0/minicluster-data/master-2/wal/instance
I20260807 20:12:34.022176 3134 external_mini_cluster.cc:1428] Setting key 5910765afc10b747bdd6eb5da9f9af8e
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:12:34.030074 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.020580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33606 (local address 127.3.15.190:33293)
0807 20:12:34.020651 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.020653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.020881 (+ 228us) server_negotiation.cc:407] Connection header received
0807 20:12:34.021345 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.021346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.021357 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.021416 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.022128 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.022741 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.024146 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.024448 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.024520 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.026940 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.026942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.026942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.026950 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.028571 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.029065 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.029067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.029067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.029137 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.029405 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.029406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.029406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.029538 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.029613 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.029893 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.029985 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:12:34.030254 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.020541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:33293 (local address 127.0.0.1:33606)
0807 20:12:34.020734 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.020793 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:34.021259 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:34.021493 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:34.021496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:34.021766 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:34.022015 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.022021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.022833 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.022834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.024064 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.024067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.024553 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.024554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.024655 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.025197 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:34.025208 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:34.026832 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:34.028657 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.028664 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.028666 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.028987 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.029207 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.029208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.029208 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.029328 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.029606 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:34.029613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:34.029753 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 20:12:34.029854 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 20:12:34.030024 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260807 20:12:34.031049 26874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:34.031910 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.022601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.3.15.188:44363)
0807 20:12:34.022786 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.022791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.022841 (+ 50us) server_negotiation.cc:407] Connection header received
0807 20:12:34.022907 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.022909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.022926 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.022981 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.023748 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.024643 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.025767 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.026056 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.026152 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.028705 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.028714 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.028714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.028721 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.030448 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.030911 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.030913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.030914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.030981 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.031221 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.031221 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.031222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.031442 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.031609 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.031683 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.031788 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:34.038283 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.031283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35827 (local address 127.0.0.1:33030)
0807 20:12:34.031362 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.031373 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:34.031411 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:34.031597 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:34.031597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:34.031779 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:34.031830 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.031832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.032515 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.032516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.033217 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.033220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.033650 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.033651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.033727 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.034063 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:34.034068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:34.035363 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:34.037147 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.037148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.037149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.037361 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.037566 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.037566 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.037567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.037620 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.037945 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:34.037946 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:34.037996 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:12:34.038074 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 20:12:34.038189 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 20:12:34.038548 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.031392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.3.15.189:35827)
0807 20:12:34.031450 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.031452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.031462 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:34.031506 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.031507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.031517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.031583 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.031945 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.032434 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.033299 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.033578 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.033631 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.035451 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.035453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.035454 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.035463 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.037065 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.037436 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.037437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.037437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.037493 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.037694 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.037696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.037697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.037843 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.037914 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.038132 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.038478 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:12:34.038944 26874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:34.040268 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.032817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.3.15.190:33293)
0807 20:12:34.032851 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.032852 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.032860 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:34.032907 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.032908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.032917 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.032981 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.033449 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.034079 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.034831 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.035028 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.035084 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.037365 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.037366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.037366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.037372 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.039160 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.039586 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.039588 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.039588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.039652 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.039854 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.039854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.039855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.039949 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.040074 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.040151 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.040209 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:12:34.046241 26876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.039185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:45008)
0807 20:12:34.039232 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.039242 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:34.039279 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:34.039405 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:34.039405 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:34.039513 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:34.039565 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.039566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.040508 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.040509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.041295 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.041297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.041706 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.041707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.041757 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.042166 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:34.042170 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:34.043366 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:34.045178 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.045178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.045179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.045342 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.045549 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.045549 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.045550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.045642 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.045875 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:34.045876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:34.045926 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:12:34.046009 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:12:34.046109 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 20:12:34.046335 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.039233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45008 (local address 127.3.15.188:44363)
0807 20:12:34.039271 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.039274 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.039281 (+ 7us) server_negotiation.cc:407] Connection header received
0807 20:12:34.039329 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.039330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.039342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.039402 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.039638 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.040317 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.041372 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.041620 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.041705 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.043454 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.043455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.043456 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.043462 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.045085 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.045414 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.045415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.045416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.045479 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.045710 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.045710 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.045711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.045808 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.045855 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.046034 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.046150 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":19}
I20260807 20:12:34.048018 26874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: Bootstrap starting.
I20260807 20:12:34.049167 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.041469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33040 (local address 127.3.15.189:35827)
0807 20:12:34.041508 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.041509 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.041554 (+ 45us) server_negotiation.cc:407] Connection header received
0807 20:12:34.041626 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.041628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.041636 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.041679 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.042284 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.042905 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.043854 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.044083 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.044155 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.046701 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.046703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.046703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.046710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.048000 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.048379 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.048379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.048380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.048438 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.048676 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.048676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.048676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.048764 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.048958 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.049036 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.049107 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 20:12:34.049234 26874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:34.049552 26874 log.cc:824] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:34.050551 26874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: No bootstrap required, opened a new log
I20260807 20:12:34.050743 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:34.052480 26874 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.052654 26874 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9f4c6a8cbd4b58a6e253f330a41ca6, State: Initialized, Role: FOLLOWER
I20260807 20:12:34.051682 26690 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "68df459d6d284e0abebdc3de0666b4d7"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33618
I20260807 20:12:34.052742 26874 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.053057 26880 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.053114 26874 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:34.053180 26880 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.054814 26718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } attempt: 2
W20260807 20:12:34.055120 26891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:34.055384 26891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:34.055378 26803 master_runner.cc:186] Error getting master registration for 127.3.15.190:33293: OK, instance_id { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" instance_seqno: 1786133553713257 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:12:34.057813 26803 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:12:34.062211 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.055088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:45012)
0807 20:12:34.055140 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.055153 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:34.055200 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:34.055329 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:34.055329 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:34.055514 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:34.055563 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.055566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.056450 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.056452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.057072 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.057074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.057526 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.057527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.057604 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.058040 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:34.058046 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:34.059502 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:34.061149 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.061150 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.061151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.061329 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.061536 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.061537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.061538 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.061606 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.061884 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:34.061886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:34.061932 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 20:12:34.062015 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 20:12:34.062118 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260807 20:12:34.062207 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.055179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45012 (local address 127.3.15.188:44363)
0807 20:12:34.055226 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.055228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.055236 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:34.055258 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.055258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.055268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.055319 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.055668 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.056385 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.057156 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.057464 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.057518 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.059590 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.059591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.059591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.059599 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.061044 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.061402 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.061403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.061404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.061464 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.061671 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.061671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.061672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.061805 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.061878 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.062042 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.062124 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:34.063423 26718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7: Bootstrap starting.
I20260807 20:12:34.064073 26718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:34.064378 26718 log.cc:824] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:34.065171 26718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7: No bootstrap required, opened a new log
I20260807 20:12:34.066188 26718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.066329 26718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68df459d6d284e0abebdc3de0666b4d7, State: Initialized, Role: FOLLOWER
I20260807 20:12:34.066447 26718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.066830 26718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:34.066797 26894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.066860 26690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 20:12:34.066885 26894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.066911 26690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:34.067802 26690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.068074 26690 leader_election.cc:288] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [CANDIDATE]: Term 1 election: Requested vote from peers 8078f38c95024f03b9d9386802d6d31f (127.3.15.189:35827), dd9f4c6a8cbd4b58a6e253f330a41ca6 (127.3.15.188:44363)
I20260807 20:12:34.068538 26845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68df459d6d284e0abebdc3de0666b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6"
I20260807 20:12:34.068672 26845 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:34.069726 26845 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68df459d6d284e0abebdc3de0666b4d7 in term 1.
I20260807 20:12:34.069947 26665 leader_election.cc:302] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68df459d6d284e0abebdc3de0666b4d7, dd9f4c6a8cbd4b58a6e253f330a41ca6; no voters:
W20260807 20:12:34.069013 26905 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:34.070143 26905 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:34.070189 26894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:34.070320 26894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [term 1 LEADER]: Becoming Leader. State: Replica: 68df459d6d284e0abebdc3de0666b4d7, State: Running, Role: LEADER
I20260807 20:12:34.070402 26894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.070843 26894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68df459d6d284e0abebdc3de0666b4d7. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.070928 26894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:34.071859 26914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:34.072268 26914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:34.071564 26647 master_runner.cc:186] Error getting master registration for 127.3.15.189:35827: OK, instance_id { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" instance_seqno: 1786133553863025 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:12:34.074380 26647 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:12:34.075336 26845 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [term 1 FOLLOWER]: Refusing update from remote peer 68df459d6d284e0abebdc3de0666b4d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:34.075695 26912 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:34.077009 26880 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68df459d6d284e0abebdc3de0666b4d7. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.077121 26880 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.078495 26912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.078622 26912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:34.078711 26915 mvcc.cc:205] Tried to move back new op lower bound from 7316003037490262016 to 7316003037473239040. Current Snapshot: MvccSnapshot[applied={T|T < 7316003037490262016}]
I20260807 20:12:34.079412 26880 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.079507 26880 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.080086 26916 mvcc.cc:205] Tried to move back new op lower bound from 7316003037490262016 to 7316003037473239040. Current Snapshot: MvccSnapshot[applied={T|T < 7316003037490262016}]
I20260807 20:12:34.080610 26914 catalog_manager.cc:1347] Generated new cluster ID: c7637b7fff064000ab5870e9c88eba50
I20260807 20:12:34.080657 26914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:34.087749 26721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.077070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.3.15.190:33293)
0807 20:12:34.077124 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.077126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.077522 (+ 396us) server_negotiation.cc:407] Connection header received
0807 20:12:34.077724 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.077726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.077739 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.077772 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.079221 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.079979 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.081737 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.082018 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.082053 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.084146 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.084147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.084148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.084154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.086078 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.086683 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.086684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.086685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.086743 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.087043 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.087044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.087045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.087183 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.087256 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.087574 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.087652 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:34.092427 26914 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:34.093640 26914 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:34.098054 26796 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } attempt: 3
I20260807 20:12:34.101323 26914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7: Generated new TSK 0
I20260807 20:12:34.101789 26914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 20:12:34.104879 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.077447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.3.15.189:35827)
0807 20:12:34.077487 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.077489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.088125 (+ 10636us) server_negotiation.cc:407] Connection header received
0807 20:12:34.088453 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.088454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.088468 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.088501 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.092085 (+ 3584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.092805 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.096069 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.096443 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.096594 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.101140 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.101142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.101142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.101155 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.103145 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.103896 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.103897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.103898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.103974 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.104198 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.104199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.104200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.104357 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.104442 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.104676 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.104760 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:34.106751 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.077248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.3.15.188:44363)
0807 20:12:34.077287 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.077289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.089172 (+ 11883us) server_negotiation.cc:407] Connection header received
0807 20:12:34.089476 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.089478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.089489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.089519 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.091247 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.091953 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.095520 (+ 3567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.095823 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.095863 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.103251 (+ 7388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.103252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.103252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.103259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.105187 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.105883 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.105884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.105884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.105948 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.106187 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.106188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.106188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.106317 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.106398 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.106582 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.106661 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 07 20:12:34 dist-test-slave-1x4g krb5kdc[26634](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133553, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:12:34.121469 26798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.098321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:44363 (local address 127.0.0.1:45042)
0807 20:12:34.098407 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:34.098417 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:34.098453 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:34.105312 (+ 6859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:34.105314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:34.105475 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:34.105523 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.105525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.106504 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.106505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.107292 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:34.107294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.115640 (+ 8346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.115642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:34.115719 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.116195 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:34.116201 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:34.118010 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:34.120194 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.120195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.120196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.120439 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.120663 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:34.120664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:34.120665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:34.120730 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:34.121116 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:34.121117 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:34.121167 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 20:12:34.121235 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 20:12:34.121355 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 20:12:34.121982 26926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:34.098524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.3.15.188:44363)
0807 20:12:34.105063 (+ 6539us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.105065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.105074 (+ 9us) server_negotiation.cc:407] Connection header received
0807 20:12:34.105095 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.105096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.105106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.105599 (+ 493us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.105633 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.106424 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.115172 (+ 8748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:34.115557 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:34.115635 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:34.118137 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:34.118139 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:34.118140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:34.118147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:34.120092 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.120522 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.120523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.120524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.120593 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:34.120857 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:34.120858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:34.120859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:34.121020 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:34.121085 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:34.121730 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 20:12:34.121823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6516,"thread_start_us":96,"threads_started":1}
I20260807 20:12:34.123657 26796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: Bootstrap starting.
I20260807 20:12:34.124441 26796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:34.124766 26796 log.cc:824] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:34.125752 26796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: No bootstrap required, opened a new log
I20260807 20:12:34.127765 26796 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.127920 26796 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8078f38c95024f03b9d9386802d6d31f, State: Initialized, Role: FOLLOWER
I20260807 20:12:34.128017 26796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } }
I20260807 20:12:34.128371 26796 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:34.128413 26770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "68df459d6d284e0abebdc3de0666b4d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8078f38c95024f03b9d9386802d6d31f"
I20260807 20:12:34.128538 26770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:34.129462 26770 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68df459d6d284e0abebdc3de0666b4d7 in term 1.
I20260807 20:12:34.129966 26769 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [term 1 FOLLOWER]: Refusing update from remote peer 68df459d6d284e0abebdc3de0666b4d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:34.130301 26894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 20:12:34.130867 26942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:34.130919 26942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:34.136880 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.137005 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.137058 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68df459d6d284e0abebdc3de0666b4d7. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.137105 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.137140 26930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.137183 26930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:34.137801 26945 mvcc.cc:205] Tried to move back new op lower bound from 7316003037591064576 to 7316003037473239040. Current Snapshot: MvccSnapshot[applied={T|T < 7316003037490262016}]
W20260807 20:12:34.147315 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.145780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33056 (local address 127.3.15.189:35827)
0807 20:12:34.145824 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.145826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.146663 (+ 837us) server_negotiation.cc:407] Connection header received
0807 20:12:34.146758 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.146759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.146772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.146807 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.147257 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33056: BlockingRecv error: recv got EOF from 127.0.0.1:33056 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:12:34.148206 26926 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.146409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.3.15.188:44363)
0807 20:12:34.146447 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.146449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.147716 (+ 1267us) server_negotiation.cc:407] Connection header received
0807 20:12:34.147799 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.147800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.147812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.147840 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.148149 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45058: BlockingRecv error: recv got EOF from 127.0.0.1:45058 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 20:12:34.148918 26721 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.146209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.3.15.190:33293)
0807 20:12:34.146259 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.146260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.147529 (+ 1269us) server_negotiation.cc:407] Connection header received
0807 20:12:34.147609 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.147610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.147621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.147683 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.148837 (+ 1154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33626: BlockingRecv error: recv got EOF from 127.0.0.1:33626 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 20:12:34.151017 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.150241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33062 (local address 127.3.15.189:35827)
0807 20:12:34.150278 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.150280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.150533 (+ 253us) server_negotiation.cc:407] Connection header received
0807 20:12:34.150636 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.150637 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.150656 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.150685 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.150975 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33062: BlockingRecv error: recv got EOF from 127.0.0.1:33062 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 20:12:34.154165 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.153320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.3.15.189:35827)
0807 20:12:34.153398 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.153399 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.153689 (+ 290us) server_negotiation.cc:407] Connection header received
0807 20:12:34.153780 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.153781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.153791 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.153820 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.154126 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33076: BlockingRecv error: recv got EOF from 127.0.0.1:33076 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 20:12:34.157945 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.157244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.3.15.189:35827)
0807 20:12:34.157276 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.157278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.157512 (+ 234us) server_negotiation.cc:407] Connection header received
0807 20:12:34.157599 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.157599 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.157608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.157635 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.157907 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33080: BlockingRecv error: recv got EOF from 127.0.0.1:33080 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260807 20:12:34.163908 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.163102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.3.15.189:35827)
0807 20:12:34.163136 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.163138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.163378 (+ 240us) server_negotiation.cc:407] Connection header received
0807 20:12:34.163493 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.163494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.163505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.163535 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.163862 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33096: BlockingRecv error: recv got EOF from 127.0.0.1:33096 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 20:12:34.174196 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.173243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33106 (local address 127.3.15.189:35827)
0807 20:12:34.173287 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.173289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.173594 (+ 305us) server_negotiation.cc:407] Connection header received
0807 20:12:34.173705 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.173707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.173718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.173749 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.174141 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33106: BlockingRecv error: recv got EOF from 127.0.0.1:33106 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 20:12:34.192595 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.191542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.3.15.189:35827)
0807 20:12:34.191585 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.191587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.191883 (+ 296us) server_negotiation.cc:407] Connection header received
0807 20:12:34.192017 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.192018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.192032 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.192065 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.192514 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33116: BlockingRecv error: recv got EOF from 127.0.0.1:33116 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:12:34.227195 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.226126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.3.15.189:35827)
0807 20:12:34.226175 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.226177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.226515 (+ 338us) server_negotiation.cc:407] Connection header received
0807 20:12:34.226629 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.226630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.226642 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.226704 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.227130 (+ 426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33126: BlockingRecv error: recv got EOF from 127.0.0.1:33126 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 20:12:34.293310 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.292604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.3.15.189:35827)
0807 20:12:34.292665 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.292667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.292746 (+ 79us) server_negotiation.cc:407] Connection header received
0807 20:12:34.292806 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.292807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.292820 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.292898 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.293248 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33130: BlockingRecv error: recv got EOF from 127.0.0.1:33130 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 20:12:34.423364 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.422493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33132 (local address 127.3.15.189:35827)
0807 20:12:34.422577 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.422580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.422682 (+ 102us) server_negotiation.cc:407] Connection header received
0807 20:12:34.422782 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.422784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.422796 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.422871 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.423302 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33132: BlockingRecv error: recv got EOF from 127.0.0.1:33132 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 20:12:34.541077 26894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "68df459d6d284e0abebdc3de0666b4d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68df459d6d284e0abebdc3de0666b4d7" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 33293 } } peers { permanent_uuid: "8078f38c95024f03b9d9386802d6d31f" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35827 } } peers { permanent_uuid: "dd9f4c6a8cbd4b58a6e253f330a41ca6" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 44363 } } }
I20260807 20:12:34.541273 26894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68df459d6d284e0abebdc3de0666b4d7 [sys.catalog]: This master's current role is: LEADER
W20260807 20:12:34.682456 26800 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:34.681652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.3.15.189:35827)
0807 20:12:34.681745 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:34.681747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:34.681771 (+ 24us) server_negotiation.cc:407] Connection header received
0807 20:12:34.681852 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:34.681853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:34.681864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:34.681919 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:34.682395 (+ 476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33140: BlockingRecv error: recv got EOF from 127.0.0.1:33140 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 20:12:35.055962 26891 catalog_manager.cc:1259] Loaded cluster ID: c7637b7fff064000ab5870e9c88eba50
I20260807 20:12:35.056041 26891 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: loading cluster ID for follower catalog manager: success
I20260807 20:12:35.056859 26891 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: acquiring CA information for follower catalog manager: success
I20260807 20:12:35.057137 26891 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd9f4c6a8cbd4b58a6e253f330a41ca6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:12:35.131361 26942 catalog_manager.cc:1259] Loaded cluster ID: c7637b7fff064000ab5870e9c88eba50
I20260807 20:12:35.131443 26942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: loading cluster ID for follower catalog manager: success
I20260807 20:12:35.132241 26942 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: acquiring CA information for follower catalog manager: success
I20260807 20:12:35.132505 26942 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8078f38c95024f03b9d9386802d6d31f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:12:35.198493 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:35.195846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.3.15.189:35827)
0807 20:12:35.195968 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:35.195970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:35.195980 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:35.196055 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:35.196056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:35.196114 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:35.196194 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:35.196391 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.196946 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.197737 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.197931 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.197986 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:35.198169 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:35.198283 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:35.198342 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:35.198379 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 20:12:35.198397 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260807 20:12:35.203400 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:35.200588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.3.15.188:44363)
0807 20:12:35.200707 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:35.200709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:35.200717 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:35.200815 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:35.200816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:35.200869 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:35.200967 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:35.201073 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.201564 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.202599 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.202796 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.202853 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:35.203015 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:35.203134 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:35.203195 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:35.203242 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 20:12:35.203262 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260807 20:12:35.208058 27022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:35.205406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.3.15.188:44363)
0807 20:12:35.205463 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:35.205465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:35.205547 (+ 82us) server_negotiation.cc:407] Connection header received
0807 20:12:35.205601 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:35.205601 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:35.205614 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:35.205667 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:35.205812 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.206512 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.207321 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.207569 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.207635 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:35.207780 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:35.207818 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:35.207869 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:35.207949 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 20:12:35.207973 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 20:12:35.208403 27015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:35.205143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.3.15.189:35827)
0807 20:12:35.205195 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:35.205197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:35.205389 (+ 192us) server_negotiation.cc:407] Connection header received
0807 20:12:35.205480 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:35.205481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:35.205493 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:35.205567 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:35.205738 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.206484 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.207495 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.207788 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.207861 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:35.208095 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:35.208133 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:35.208166 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:35.208291 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 20:12:35.208323 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:35.208607 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:35.205162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33634 (local address 127.3.15.190:33293)
0807 20:12:35.205280 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:35.205282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:35.205375 (+ 93us) server_negotiation.cc:407] Connection header received
0807 20:12:35.205569 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:35.205571 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:35.205630 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:35.205692 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:35.205888 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.206388 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.207493 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:35.207808 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:35.207849 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:35.208157 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:35.208288 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:35.208354 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:35.208462 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 20:12:35.208495 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260807 20:12:35.210369 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26647
I20260807 20:12:35.217978 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26725
I20260807 20:12:35.224340 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 26803
2026-08-07T20:12:35Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2835 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 20:12:35 dist-test-slave-1x4g krb5kdc[27033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 20:12:35 dist-test-slave-1x4g krb5kdc[27033](info): set up 2 sockets
Aug 07 20:12:35 dist-test-slave-1x4g krb5kdc[27033](info): commencing operation
krb5kdc: starting...
W20260807 20:12:36.311120 3134 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/test-user.keytab.
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T20:12:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T20:12:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.15.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
Entry for principal kudu/127.3.15.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab.
I20260807 20:12:36.440008 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39887
--webserver_interface=127.3.15.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39887,127.3.15.189:35881,127.3.15.188:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 20:12:36.523509 27046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:12:36.523803 27046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:36.523834 27046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:36.523895 27046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:36.525110 27046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:36.525148 27046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:36.525163 27046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:36.525175 27046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:36.525187 27046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:36.526625 27046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39887,127.3.15.189:35881,127.3.15.188:39895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.190.keytab
--principal=kudu/127.3.15.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.190:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.15.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:36.526824 27046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:36.527005 27046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:36.532330 27052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:36.532330 27053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:36.532635 27046 server_base.cc:1034] running on GCE node
W20260807 20:12:36.532323 27055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:36.532984 27046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:36.533273 27046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:36.534458 27046 hybrid_clock.cc:648] HybridClock initialized: now 1786133556534435 us; error 41 us; skew 500 ppm
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:36.541199 27046 init.cc:377] Logged in from keytab as kudu/127.3.15.190@KRBTEST.COM (short username kudu)
I20260807 20:12:36.541604 27046 webserver.cc:460] Webserver started at http://127.3.15.190:41219/ using document root <none> and password file <none>
I20260807 20:12:36.541859 27046 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:36.541931 27046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:36.542094 27046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:36.543177 27046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data/instance:
uuid: "f9722e0dc6ad4cae85a744510715e54d"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "d1053bcc73815017c3e3fe4385db68df"
server_key_iv: "b61220833ea0b809d703e63a571300ac"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.543550 27046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal/instance:
uuid: "f9722e0dc6ad4cae85a744510715e54d"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "d1053bcc73815017c3e3fe4385db68df"
server_key_iv: "b61220833ea0b809d703e63a571300ac"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.545084 27046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 20:12:36.545940 27063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:36.546183 27046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 20:12:36.546291 27046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal
uuid: "f9722e0dc6ad4cae85a744510715e54d"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "d1053bcc73815017c3e3fe4385db68df"
server_key_iv: "b61220833ea0b809d703e63a571300ac"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.546387 27046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:36.558877 27046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:36.560611 27046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:36.560820 27046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:36.564751 27046 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.190:39887
I20260807 20:12:36.564834 27117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.190:39887 every 8 connection(s)
I20260807 20:12:36.565269 27046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/data/info.pb
I20260807 20:12:36.566265 27118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:36.567847 27118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:36.575482 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 27046
I20260807 20:12:36.575567 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-0/wal/instance
I20260807 20:12:36.575789 3134 external_mini_cluster.cc:1428] Setting key fb2f11e659ab7a3de9c9d469aff142f5
I20260807 20:12:36.577214 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.568497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39887 (local address 127.0.0.1:60654)
0807 20:12:36.568648 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.568682 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.569174 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.569433 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.569436 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.569678 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.569830 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.569834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.570875 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.570875 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.571552 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.571554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.572017 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.572018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.572077 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.572590 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.572600 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.574056 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.575643 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.575649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.575651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.575842 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.576111 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.576112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.576112 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.576202 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.576567 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.576572 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.576645 (+ 73us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.576754 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.577031 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":108,"thread_start_us":68,"threads_started":1}
I20260807 20:12:36.577221 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.568646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60654 (local address 127.3.15.190:39887)
0807 20:12:36.568789 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.568793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.568833 (+ 40us) server_negotiation.cc:407] Connection header received
0807 20:12:36.569309 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.569311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.569327 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.569402 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.569953 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.570789 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.571628 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.571926 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.572002 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.574136 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.574144 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.574144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.574150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.575553 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.575957 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.575964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.575964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.576037 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.576269 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.576270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.576270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.576491 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.576587 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.576805 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.576985 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260807 20:12:36.577955 27118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:36.578294 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.578177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43068)
0807 20:12:36.578222 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.578264 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:12:36.578459 27066 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
I20260807 20:12:36.578892 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.578805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43082)
0807 20:12:36.578852 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.578877 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 20:12:36.579082 27118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:35881: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.15.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab.
Entry for principal kudu/127.3.15.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab.
I20260807 20:12:36.601873 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:35881
--webserver_interface=127.3.15.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab
--principal=kudu/127.3.15.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39887,127.3.15.189:35881,127.3.15.188:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:12:36.628327 27118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } attempt: 1
I20260807 20:12:36.628895 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.628739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43086)
0807 20:12:36.628814 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.628861 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 20:12:36.629297 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.629194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43098)
0807 20:12:36.629243 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.629273 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:12:36.629432 27118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:35881: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
W20260807 20:12:36.685437 27125 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:12:36.685740 27125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:36.685774 27125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:36.685817 27125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:36.686923 27125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:36.686982 27125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:36.687000 27125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:36.687050 27125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:36.687070 27125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:36.688541 27125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39887,127.3.15.189:35881,127.3.15.188:39895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.189.keytab
--principal=kudu/127.3.15.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.189:35881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.15.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:36.688835 27125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:36.689061 27125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:36.693962 27130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:36.693941 27131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:36.694159 27125 server_base.cc:1034] running on GCE node
W20260807 20:12:36.693938 27133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:36.694448 27125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:36.694757 27125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:36.695907 27125 hybrid_clock.cc:648] HybridClock initialized: now 1786133556695899 us; error 27 us; skew 500 ppm
I20260807 20:12:36.697623 27118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } attempt: 2
I20260807 20:12:36.698009 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.697877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43106)
0807 20:12:36.697939 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.697975 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:36.698386 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.698315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43118)
0807 20:12:36.698344 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.698369 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 20:12:36.698503 27118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.189:35881: Network error: Client connection negotiation failed: client connection to 127.3.15.189:35881: connect: Connection refused (error 111)
I20260807 20:12:36.698700 27125 init.cc:377] Logged in from keytab as kudu/127.3.15.189@KRBTEST.COM (short username kudu)
I20260807 20:12:36.699132 27125 webserver.cc:460] Webserver started at http://127.3.15.189:41929/ using document root <none> and password file <none>
I20260807 20:12:36.699383 27125 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:36.699426 27125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:36.699569 27125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:36.700611 27125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data/instance:
uuid: "624c1391b6094af1ad57c9c875236497"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "148c084511d83dd0d09d5c7f73cae01b"
server_key_iv: "f00406e7c6de4fd50519da1063cfda05"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.700995 27125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal/instance:
uuid: "624c1391b6094af1ad57c9c875236497"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "148c084511d83dd0d09d5c7f73cae01b"
server_key_iv: "f00406e7c6de4fd50519da1063cfda05"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.702275 27125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 20:12:36.702963 27142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:36.703118 27125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:36.703204 27125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal
uuid: "624c1391b6094af1ad57c9c875236497"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "148c084511d83dd0d09d5c7f73cae01b"
server_key_iv: "f00406e7c6de4fd50519da1063cfda05"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.703258 27125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:36.713440 27125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:36.715246 27125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:36.715353 27125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:36.719252 27125 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.189:35881
I20260807 20:12:36.719321 27196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.189:35881 every 8 connection(s)
I20260807 20:12:36.719668 27125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/data/info.pb
I20260807 20:12:36.720729 27197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:36.722347 27197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:36.726850 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 27125
I20260807 20:12:36.726933 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-1/wal/instance
I20260807 20:12:36.727133 3134 external_mini_cluster.cc:1428] Setting key 3ea6226f3bf217fafab7765559e0ca31
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:36.731526 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.723048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60668 (local address 127.3.15.190:39887)
0807 20:12:36.723119 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.723121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.723299 (+ 178us) server_negotiation.cc:407] Connection header received
0807 20:12:36.723738 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.723739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.723748 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.723806 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.724512 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.725004 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.725963 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.726162 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.726215 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.728434 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.728436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.728436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.728444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.730150 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.730537 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.730538 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.730539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.730595 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.730864 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.730865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.730865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.730986 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.731060 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.731349 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.731435 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 20:12:36.731719 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.723003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39887 (local address 127.0.0.1:60668)
0807 20:12:36.723171 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.723215 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.723660 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.723886 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.723889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.724228 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.724431 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.724435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.725080 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.725080 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.725891 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.725894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.726246 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.726247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.726351 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.726868 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.726877 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.728330 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.730237 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.730244 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.730246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.730468 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.730672 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.730674 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.730674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.730792 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.731065 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.731072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.731210 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.731314 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.731471 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260807 20:12:36.732405 27197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.15.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.15.188@KRBTEST.COM" created.
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:12:36.742751 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.732683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43126)
0807 20:12:36.732737 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.732749 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.732810 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.733080 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.733081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.733227 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.733270 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.733271 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.734129 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.734130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.734638 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.734639 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.735122 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.735123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.735187 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.735656 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.735661 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.739595 (+ 3934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.741471 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.741472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.741472 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.741643 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.741938 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.741939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.741939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.742012 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.742314 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.742315 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.742366 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.742441 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.742563 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 20:12:36.742763 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.732772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.3.15.189:35881)
0807 20:12:36.732939 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.732944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.732968 (+ 24us) server_negotiation.cc:407] Connection header received
0807 20:12:36.732986 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.732987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.732998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.733079 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.733344 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.734010 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.734719 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.735027 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.735123 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.739693 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.739704 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.739704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.739711 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.741365 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.741747 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.741751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.741751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.741857 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.742080 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.742081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.742081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.742246 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.742315 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.742479 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.742650 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":77,"server-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
I20260807 20:12:36.743589 27197 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:36.743958 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.743824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54532)
0807 20:12:36.743874 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.743927 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 20:12:36.744127 27144 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
I20260807 20:12:36.744594 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.744480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54538)
0807 20:12:36.744530 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.744563 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 20:12:36.744720 27197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:39895: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.15.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab.
Entry for principal kudu/127.3.15.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab.
I20260807 20:12:36.755286 3134 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
/tmp/dist-test-taskp_X5oZ/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:39895
--webserver_interface=127.3.15.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab
--principal=kudu/127.3.15.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.15.148:45601
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.15.190:39887,127.3.15.189:35881,127.3.15.188:39895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 20:12:36.789676 27118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } attempt: 3
I20260807 20:12:36.793988 27197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } attempt: 1
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:12:36.794402 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.794274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54554)
0807 20:12:36.794339 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.794372 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 20:12:36.794821 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.794715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54566)
0807 20:12:36.794762 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.794793 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 20:12:36.795095 27197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:39895: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
I20260807 20:12:36.797232 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.790082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43128)
0807 20:12:36.790164 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.790190 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.790243 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.790403 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.790404 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.790573 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.790645 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.790646 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.791346 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.791347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.791884 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.791886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.792318 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.792319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.792407 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.792990 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.792995 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.794441 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.796257 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.796258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.796259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.796432 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.796644 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.796644 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.796645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.796701 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.796985 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.796985 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.797028 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.797084 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.797172 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 20:12:36.797290 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.790154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.3.15.189:35881)
0807 20:12:36.790227 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.790230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.790242 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:36.790305 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.790306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.790325 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.790384 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.790741 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.791253 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.791981 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.792221 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.792314 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.794548 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.794549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.794550 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.794558 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.796171 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.796500 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.796501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.796501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.796564 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.796761 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.796761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.796762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.796920 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.796990 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.797128 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.797208 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 20:12:36.797664 27118 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:36.797968 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.797885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54576)
0807 20:12:36.797917 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.797947 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260807 20:12:36.798269 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.798216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54580)
0807 20:12:36.798237 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.798257 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260807 20:12:36.798367 27118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:39895: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
I20260807 20:12:36.829538 27118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } attempt: 1
I20260807 20:12:36.830125 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.829954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54596)
0807 20:12:36.830034 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.830084 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 20:12:36.831094 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.830938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54600)
0807 20:12:36.830996 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.831025 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260807 20:12:36.831316 27118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:39895: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
W20260807 20:12:36.844542 27205 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 20:12:36.844763 27205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 20:12:36.844841 27205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 20:12:36.844871 27205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 20:12:36.845990 27205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 20:12:36.846032 27205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 20:12:36.846047 27205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 20:12:36.846060 27205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 20:12:36.846072 27205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 20:12:36.847594 27205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.15.148:45601
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.15.190:39887,127.3.15.189:35881,127.3.15.188:39895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/krb5kdc/kudu_127.3.15.188.keytab
--principal=kudu/127.3.15.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.15.188:39895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.15.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 18:57:08 UTC on ccycloud.quasar-aagcri.root.comops.site
build id 4573429
I20260807 20:12:36.847790 27205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 20:12:36.847954 27205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 20:12:36.853011 27210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:36.853008 27213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 20:12:36.853055 27211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 20:12:36.853226 27205 server_base.cc:1034] running on GCE node
I20260807 20:12:36.853520 27205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 20:12:36.853772 27205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 20:12:36.854911 27205 hybrid_clock.cc:648] HybridClock initialized: now 1786133556854889 us; error 44 us; skew 500 ppm
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 20:12:36.857530 27205 init.cc:377] Logged in from keytab as kudu/127.3.15.188@KRBTEST.COM (short username kudu)
I20260807 20:12:36.858026 27205 webserver.cc:460] Webserver started at http://127.3.15.188:44733/ using document root <none> and password file <none>
I20260807 20:12:36.858258 27205 fs_manager.cc:359] Metadata directory not provided
I20260807 20:12:36.858323 27205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 20:12:36.858471 27205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 20:12:36.859529 27205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data/instance:
uuid: "5d93525b095e409a9fe15195622f1123"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "f819c42f222a388f0f0a2123b0a90b2d"
server_key_iv: "a3a0fe9f686065023a7f995f747ef217"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.859905 27205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal/instance:
uuid: "5d93525b095e409a9fe15195622f1123"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "f819c42f222a388f0f0a2123b0a90b2d"
server_key_iv: "a3a0fe9f686065023a7f995f747ef217"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.861470 27205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 20:12:36.862196 27220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 20:12:36.862370 27205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 20:12:36.862471 27205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data,/tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal
uuid: "5d93525b095e409a9fe15195622f1123"
format_stamp: "Formatted at 2026-08-07 20:12:36 on dist-test-slave-1x4g"
server_key: "f819c42f222a388f0f0a2123b0a90b2d"
server_key_iv: "a3a0fe9f686065023a7f995f747ef217"
server_key_version: "encryptionkey@0"
I20260807 20:12:36.862550 27205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 20:12:36.863265 27197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } attempt: 2
I20260807 20:12:36.863679 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.863543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54612)
0807 20:12:36.863622 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.863651 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 20:12:36.864018 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.863922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54618)
0807 20:12:36.863967 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.863991 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 20:12:36.864114 27197 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:39895: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
I20260807 20:12:36.903119 27205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 20:12:36.905072 27205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 20:12:36.905236 27205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 20:12:36.906455 27118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } attempt: 2
I20260807 20:12:36.906893 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.906721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54630)
0807 20:12:36.906802 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.906850 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 20:12:36.907325 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.907205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54636)
0807 20:12:36.907252 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.907295 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260807 20:12:36.907426 27118 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.15.188:39895: Network error: Client connection negotiation failed: client connection to 127.3.15.188:39895: connect: Connection refused (error 111)
I20260807 20:12:36.909507 27205 rpc_server.cc:307] RPC server started. Bound to: 127.3.15.188:39895
I20260807 20:12:36.909546 27273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.15.188:39895 every 8 connection(s)
I20260807 20:12:36.909926 27205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/data/info.pb
I20260807 20:12:36.910909 27274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 20:12:36.911353 3134 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu as pid 27205
I20260807 20:12:36.911473 3134 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskp_X5oZ/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786133281907902-3134-0/minicluster-data/master-2/wal/instance
I20260807 20:12:36.911679 3134 external_mini_cluster.cc:1428] Setting key d233ee05080012a525200b099a832107
I20260807 20:12:36.912873 27274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:12:36.922617 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.913565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.3.15.190:39887)
0807 20:12:36.913610 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.913612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.913719 (+ 107us) server_negotiation.cc:407] Connection header received
0807 20:12:36.913803 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.913804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.913831 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.913896 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.914348 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.915046 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.916297 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.916573 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.916663 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.919264 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.919266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.919267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.919275 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.921209 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.921639 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.921640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.921641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.921705 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.921951 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.921952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.921953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.922084 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.922162 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.922438 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.922522 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 20:12:36.922950 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.913443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.190:39887 (local address 127.0.0.1:60684)
0807 20:12:36.913619 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.913663 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.913742 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.913917 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.913919 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.914167 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.914257 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.914261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.915125 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.915126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.916217 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.916219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.916689 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.916690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.916818 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.917473 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.917485 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.919170 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.921297 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.921304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.921306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.921563 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.921776 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.921777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.921777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.921885 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.922153 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.922159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.922301 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.922416 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.922582 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260807 20:12:36.923130 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.912180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54642 (local address 127.3.15.188:39895)
0807 20:12:36.912372 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.912381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.912407 (+ 26us) server_negotiation.cc:407] Connection header received
0807 20:12:36.912878 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.912886 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.912937 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.913082 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.913870 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.915158 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.916196 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.916682 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.916819 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.919874 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.919882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.919883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.919891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.921703 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.922102 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.922106 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.922106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.922169 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.922378 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.922379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.922380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.922527 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.922819 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.922868 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.922947 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260807 20:12:36.923647 27274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.190@KRBTEST.COM
I20260807 20:12:36.930657 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.923905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.189:35881 (local address 127.0.0.1:43140)
0807 20:12:36.923958 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.923971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.924015 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.924189 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.924190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.924329 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.924375 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.924377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.925182 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.925183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.925675 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.925676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.926074 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.926075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.926153 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.926500 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.926504 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.927851 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.929714 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.929715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.929716 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.929876 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.930082 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.930082 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.930083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.930126 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.930367 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.930368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.930411 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.930491 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.930587 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 20:12:36.930742 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.924029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.3.15.189:35881)
0807 20:12:36.924073 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.924075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.924083 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:36.924107 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.924108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.924117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.924185 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.924459 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.925096 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.925762 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.925985 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.926058 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.927937 (+ 1879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.927939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.927939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.927948 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.929627 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.929949 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.929950 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.929951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.930015 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.930163 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.930163 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.930164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.930300 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.930358 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.930517 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.930627 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:36.931083 27274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } has no permanent_uuid. Determining permanent_uuid...
I20260807 20:12:36.931511 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.923973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60694 (local address 127.3.15.190:39887)
0807 20:12:36.924011 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.924012 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.924022 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:36.924055 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.924056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.924064 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.924112 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.924667 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.925190 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.925838 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.926045 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.926119 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.928817 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.928819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.928819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.928827 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.930198 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.930572 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.930573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.930573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.930629 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.930859 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.930860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.930861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.930995 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.931176 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.931246 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.931328 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.188@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.15.189@KRBTEST.COM
I20260807 20:12:36.937748 27278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.931331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54654)
0807 20:12:36.931378 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.931387 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.931424 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.931534 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.931534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.931666 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.931693 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.931694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.932447 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.932448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.932967 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.932969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.933356 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.933357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.933422 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.933802 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.933807 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.935017 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.936707 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.936707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.936708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.936932 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.937143 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.937144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.937144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.937197 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.937442 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.937443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.937479 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.937546 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.937641 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 20:12:36.937805 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.931396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54654 (local address 127.3.15.188:39895)
0807 20:12:36.931430 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.931432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.931442 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:36.931483 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.931483 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.931491 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.931528 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.931788 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.932359 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.933062 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.933276 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.933347 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.935106 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.935107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.935108 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.935115 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.936606 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.937014 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.937015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.937016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.937068 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.937267 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.937268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.937268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.937376 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.937449 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.937577 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.937692 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":20}
I20260807 20:12:36.938966 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.932190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43142 (local address 127.3.15.189:35881)
0807 20:12:36.932229 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.932231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.932239 (+ 8us) server_negotiation.cc:407] Connection header received
0807 20:12:36.932262 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.932263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.932271 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.932330 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.932796 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.933232 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.933880 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.934092 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.934165 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.936332 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.936334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.936334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.936341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.937848 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.938257 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.938258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.938258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.938318 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.938532 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.938533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.938533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.938642 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.938757 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.938857 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.938910 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:36.939915 27274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: Bootstrap starting.
I20260807 20:12:36.940620 3134 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 20:12:36.941000 27274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:36.941351 27274 log.cc:824] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:36.942193 27274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: No bootstrap required, opened a new log
I20260807 20:12:36.941699 27090 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f9722e0dc6ad4cae85a744510715e54d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60694
I20260807 20:12:36.944154 27274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:36.944311 27274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d93525b095e409a9fe15195622f1123, State: Initialized, Role: FOLLOWER
I20260807 20:12:36.944388 27274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:36.944655 27280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:36.944718 27274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:36.944792 27280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:12:36.946591 27291 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:36.946652 27291 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:36.946928 27205 master_runner.cc:186] Error getting master registration for 127.3.15.190:39887: OK, instance_id { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" instance_seqno: 1786133556563720 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:12:36.949295 27205 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:12:36.955251 27197 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } attempt: 3
Aug 07 20:12:36 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.189@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:12:36.962618 27199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.955550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54656)
0807 20:12:36.955600 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:36.955612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:36.955666 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:36.955808 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:36.955809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:36.955969 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:36.956011 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.956012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.956965 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.956966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.957467 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:36.957469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.957887 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.957889 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:36.957955 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.958408 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:36.958413 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:36.959692 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:36.961486 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.961487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.961488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.961714 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.961921 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:36.961921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:36.961922 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:36.961988 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:36.962293 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:36.962294 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:36.962343 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 20:12:36.962418 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 20:12:36.962515 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 20:12:36.962628 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:36.955617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54656 (local address 127.3.15.188:39895)
0807 20:12:36.955656 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:36.955658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:36.955668 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:36.955731 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:36.955732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:36.955743 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:36.955794 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:36.956100 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.956889 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.957600 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:36.957810 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:36.957888 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:36.959780 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:36.959781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:36.959781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:36.959787 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:36.961399 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.961792 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.961793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.961794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.961853 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:36.962054 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:36.962055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:36.962055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:36.962221 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:36.962287 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:36.962453 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 20:12:36.962530 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:36.964499 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: Bootstrap starting.
I20260807 20:12:36.965294 27197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:36.965668 27197 log.cc:824] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:36.966506 27197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: No bootstrap required, opened a new log
I20260807 20:12:36.967932 27197 raft_consensus.cc:348] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:36.968075 27197 raft_consensus.cc:729] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624c1391b6094af1ad57c9c875236497, State: Initialized, Role: FOLLOWER
I20260807 20:12:36.968166 27197 consensus_queue.cc:260] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:36.968441 27294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:36.968519 27197 sys_catalog.cc:564] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:36.968590 27294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 20:12:36.970557 27306 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:36.970613 27306 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:36.970599 27125 master_runner.cc:186] Error getting master registration for 127.3.15.190:39887: OK, instance_id { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" instance_seqno: 1786133556563720 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 20:12:36.972941 27125 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:12:37.006589 27118 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } attempt: 3
Aug 07 20:12:37 dist-test-slave-1x4g krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786133556, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.15.190@KRBTEST.COM for kudu/127.3.15.188@KRBTEST.COM
I20260807 20:12:37.015990 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:37.007212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54668 (local address 127.3.15.188:39895)
0807 20:12:37.007289 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.007292 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.007302 (+ 10us) server_negotiation.cc:407] Connection header received
0807 20:12:37.007335 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.007336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.007347 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.007425 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.007817 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.008589 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.009686 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.010004 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.010079 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:37.012434 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:37.012435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:37.012436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:37.012445 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:37.014743 (+ 2298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.015172 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.015173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.015173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.015234 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.015447 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.015448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.015448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.015584 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:37.015651 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:37.015823 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 20:12:37.015906 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 20:12:37.015990 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:37.006978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.15.188:39895 (local address 127.0.0.1:54668)
0807 20:12:37.007049 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 20:12:37.007061 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 20:12:37.007105 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 20:12:37.007461 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 20:12:37.007462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 20:12:37.007674 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 20:12:37.007737 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:37.007739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.008686 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.008687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:37.009569 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 20:12:37.009572 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.010142 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.010143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 20:12:37.010201 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:37.010727 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 20:12:37.010732 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 20:12:37.012345 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 20:12:37.014837 (+ 2492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:37.014838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:37.014839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:37.015076 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:37.015304 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 20:12:37.015305 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 20:12:37.015305 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 20:12:37.015374 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 20:12:37.015654 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 20:12:37.015655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 20:12:37.015708 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 20:12:37.015775 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 20:12:37.015879 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 20:12:37.018172 27118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d: Bootstrap starting.
I20260807 20:12:37.018949 27118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d: Neither blocks nor log segments found. Creating new log.
I20260807 20:12:37.019284 27118 log.cc:824] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d: Log is configured to *not* fsync() on all Append() calls
I20260807 20:12:37.020304 27118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d: No bootstrap required, opened a new log
I20260807 20:12:37.021719 27118 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:37.021867 27118 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9722e0dc6ad4cae85a744510715e54d, State: Initialized, Role: FOLLOWER
I20260807 20:12:37.021945 27118 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:37.022259 27118 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 20:12:37.022308 27090 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 20:12:37.022378 27090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:37.023171 27090 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:37.023422 27090 leader_election.cc:288] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [CANDIDATE]: Term 1 election: Requested vote from peers 624c1391b6094af1ad57c9c875236497 (127.3.15.189:35881), 5d93525b095e409a9fe15195622f1123 (127.3.15.188:39895)
I20260807 20:12:37.023738 27170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f9722e0dc6ad4cae85a744510715e54d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "624c1391b6094af1ad57c9c875236497"
I20260807 20:12:37.023874 27170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [term 0 FOLLOWER]: Advancing to term 1
I20260807 20:12:37.024748 27170 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9722e0dc6ad4cae85a744510715e54d in term 1.
I20260807 20:12:37.025048 27066 leader_election.cc:302] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 624c1391b6094af1ad57c9c875236497, f9722e0dc6ad4cae85a744510715e54d; no voters:
I20260807 20:12:37.025101 27247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f9722e0dc6ad4cae85a744510715e54d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5d93525b095e409a9fe15195622f1123"
I20260807 20:12:37.025215 27247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [term 0 FOLLOWER]: Advancing to term 1
W20260807 20:12:37.025913 27325 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 20:12:37.025960 27325 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 20:12:37.026054 27247 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9722e0dc6ad4cae85a744510715e54d in term 1.
I20260807 20:12:37.026223 27322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 20:12:37.026368 27322 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [term 1 LEADER]: Becoming Leader. State: Replica: f9722e0dc6ad4cae85a744510715e54d, State: Running, Role: LEADER
I20260807 20:12:37.026454 27322 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } }
I20260807 20:12:37.022258 27309 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.026747 27309 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:37.026885 27309 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9722e0dc6ad4cae85a744510715e54d. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.027004 27309 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:37.028100 27046 master_runner.cc:263] No leader master found from master f9722e0dc6ad4cae85a744510715e54d
I20260807 20:12:37.028146 27046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 20:12:37.029070 27329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 20:12:37.029289 27329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 20:12:37.045944 27201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:37.028473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.3.15.189:35881)
0807 20:12:37.028513 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.028515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.031157 (+ 2642us) server_negotiation.cc:407] Connection header received
0807 20:12:37.031247 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.031248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.031258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.031287 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.033788 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.034430 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.036305 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.036597 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.036658 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:37.039402 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:37.039404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:37.039404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:37.039412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:37.041251 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.044456 (+ 3205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.044458 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.044458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.044540 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.044833 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.044834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.044834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.044987 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:37.045076 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:37.045715 (+ 639us) server_negotiation.cc:299] Negotiation successful
0807 20:12:37.045819 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 20:12:37.048009 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:37.027372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.3.15.190:39887)
0807 20:12:37.027487 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.027489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.030163 (+ 2674us) server_negotiation.cc:407] Connection header received
0807 20:12:37.030261 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.030262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.030272 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.030302 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.031977 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.032717 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.035424 (+ 2707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.035710 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.035750 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:37.041685 (+ 5935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:37.041687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:37.041687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:37.041695 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:37.043478 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.045296 (+ 1818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.045298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.045298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.045359 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.047067 (+ 1708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.047068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.047069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.047212 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:37.047311 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:37.047813 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 20:12:37.047904 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 20:12:37.049104 27247 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [term 1 FOLLOWER]: Refusing update from remote peer f9722e0dc6ad4cae85a744510715e54d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:37.049535 27327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:37.051730 27280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9722e0dc6ad4cae85a744510715e54d. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.051826 27280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:37.052322 27170 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [term 1 FOLLOWER]: Refusing update from remote peer f9722e0dc6ad4cae85a744510715e54d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 20:12:37.052623 27327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [LEADER]: Connected to new peer: Peer: permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 20:12:37.052881 27277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:37.028805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54674 (local address 127.3.15.188:39895)
0807 20:12:37.028841 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.028843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.032969 (+ 4126us) server_negotiation.cc:407] Connection header received
0807 20:12:37.033078 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.033079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.033089 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.033118 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.044868 (+ 11750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.045637 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.047843 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:37.048148 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:37.048189 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:37.049817 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 20:12:37.049819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 20:12:37.049819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 20:12:37.049827 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 20:12:37.051436 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.051815 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.051816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.051817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.051881 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 20:12:37.052159 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 20:12:37.052160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 20:12:37.052160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 20:12:37.052280 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 20:12:37.052381 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:37.052697 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 20:12:37.052767 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 20:12:37.053853 27294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9722e0dc6ad4cae85a744510715e54d. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.053952 27294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:37.055994 27327 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.056092 27327 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:37.056659 27294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.056740 27294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:37.057030 27333 mvcc.cc:205] Tried to move back new op lower bound from 7316003049652969472 to 7316003049581113344. Current Snapshot: MvccSnapshot[applied={T|T < 7316003049652969472}]
I20260807 20:12:37.057189 27336 mvcc.cc:205] Tried to move back new op lower bound from 7316003049652969472 to 7316003049581113344. Current Snapshot: MvccSnapshot[applied={T|T < 7316003049652969472}]
I20260807 20:12:37.059083 27322 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.059163 27322 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d [sys.catalog]: This master's current role is: LEADER
I20260807 20:12:37.059593 27280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f9722e0dc6ad4cae85a744510715e54d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9722e0dc6ad4cae85a744510715e54d" member_type: VOTER last_known_addr { host: "127.3.15.190" port: 39887 } } peers { permanent_uuid: "624c1391b6094af1ad57c9c875236497" member_type: VOTER last_known_addr { host: "127.3.15.189" port: 35881 } } peers { permanent_uuid: "5d93525b095e409a9fe15195622f1123" member_type: VOTER last_known_addr { host: "127.3.15.188" port: 39895 } } }
I20260807 20:12:37.059672 27280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 20:12:37.060364 27334 mvcc.cc:205] Tried to move back new op lower bound from 7316003049652969472 to 7316003049581113344. Current Snapshot: MvccSnapshot[applied={T|T < 7316003049652969472}]
I20260807 20:12:37.062310 27329 catalog_manager.cc:1347] Generated new cluster ID: 425ce399867041a29c9f6d6c3f06680e
I20260807 20:12:37.062362 27329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 20:12:37.079741 27329 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 20:12:37.080530 27329 catalog_manager.cc:1504] Loading token signing keys...
I20260807 20:12:37.089102 27329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9722e0dc6ad4cae85a744510715e54d: Generated new TSK 0
I20260807 20:12:37.089354 27329 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 20:12:37.111166 27121 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.109268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60708 (local address 127.3.15.190:39887)
0807 20:12:37.109317 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.109320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.110405 (+ 1085us) server_negotiation.cc:407] Connection header received
0807 20:12:37.110523 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.110524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.110537 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.110569 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.111090 (+ 521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60708: BlockingRecv error: recv got EOF from 127.0.0.1:60708 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 20:12:37.112154 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.109699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43150 (local address 127.3.15.189:35881)
0807 20:12:37.109739 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.109741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.111606 (+ 1865us) server_negotiation.cc:407] Connection header received
0807 20:12:37.111724 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.111726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.111736 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.111765 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.112098 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43150: BlockingRecv error: recv got EOF from 127.0.0.1:43150 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 20:12:37.113176 27277 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.110118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54682 (local address 127.3.15.188:39895)
0807 20:12:37.110175 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.110177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.112506 (+ 2329us) server_negotiation.cc:407] Connection header received
0807 20:12:37.112657 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.112659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.112670 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.112700 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.113119 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54682: BlockingRecv error: recv got EOF from 127.0.0.1:54682 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260807 20:12:37.122083 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.121108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.3.15.189:35881)
0807 20:12:37.121155 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.121157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.121441 (+ 284us) server_negotiation.cc:407] Connection header received
0807 20:12:37.121565 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.121567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.121586 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.121618 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.122025 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43160: BlockingRecv error: recv got EOF from 127.0.0.1:43160 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 20:12:37.125905 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.125043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43176 (local address 127.3.15.189:35881)
0807 20:12:37.125110 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.125112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.125360 (+ 248us) server_negotiation.cc:407] Connection header received
0807 20:12:37.125461 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.125462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.125475 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.125506 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.125853 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43176: BlockingRecv error: recv got EOF from 127.0.0.1:43176 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260807 20:12:37.130054 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.129248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.3.15.189:35881)
0807 20:12:37.129283 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.129285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.129516 (+ 231us) server_negotiation.cc:407] Connection header received
0807 20:12:37.129616 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.129616 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.129626 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.129655 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.130006 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43178: BlockingRecv error: recv got EOF from 127.0.0.1:43178 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:12:37.136026 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.135252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43190 (local address 127.3.15.189:35881)
0807 20:12:37.135285 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.135286 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.135507 (+ 221us) server_negotiation.cc:407] Connection header received
0807 20:12:37.135605 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.135607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.135619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.135652 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.135977 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43190: BlockingRecv error: recv got EOF from 127.0.0.1:43190 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 20:12:37.146449 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.145538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43198 (local address 127.3.15.189:35881)
0807 20:12:37.145604 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.145607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.145746 (+ 139us) server_negotiation.cc:407] Connection header received
0807 20:12:37.145865 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.145866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.145879 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.145951 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.146385 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43198: BlockingRecv error: recv got EOF from 127.0.0.1:43198 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 20:12:37.164178 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.163467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43204 (local address 127.3.15.189:35881)
0807 20:12:37.163518 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.163520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.163628 (+ 108us) server_negotiation.cc:407] Connection header received
0807 20:12:37.163670 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.163671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.163686 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.163790 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.164120 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43204: BlockingRecv error: recv got EOF from 127.0.0.1:43204 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 20:12:37.198280 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.197535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.3.15.189:35881)
0807 20:12:37.197626 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.197628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.197676 (+ 48us) server_negotiation.cc:407] Connection header received
0807 20:12:37.197738 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.197739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.197750 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.197793 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.198212 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43214: BlockingRecv error: recv got EOF from 127.0.0.1:43214 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 20:12:37.264289 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.263487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43220 (local address 127.3.15.189:35881)
0807 20:12:37.263567 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.263570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.263680 (+ 110us) server_negotiation.cc:407] Connection header received
0807 20:12:37.263784 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.263785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.263797 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.263871 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.264219 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43220: BlockingRecv error: recv got EOF from 127.0.0.1:43220 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 20:12:37.394392 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.393601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.3.15.189:35881)
0807 20:12:37.393692 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.393695 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.393795 (+ 100us) server_negotiation.cc:407] Connection header received
0807 20:12:37.393900 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.393901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.393914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.393961 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.394324 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43224: BlockingRecv error: recv got EOF from 127.0.0.1:43224 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260807 20:12:37.652259 27201 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 20:12:37.651599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43240 (local address 127.3.15.189:35881)
0807 20:12:37.651670 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:37.651672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:37.651684 (+ 12us) server_negotiation.cc:407] Connection header received
0807 20:12:37.651744 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:37.651745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:37.651756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:37.651808 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 20:12:37.652168 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43240: BlockingRecv error: recv got EOF from 127.0.0.1:43240 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 20:12:37.947117 27291 catalog_manager.cc:1259] Loaded cluster ID: 425ce399867041a29c9f6d6c3f06680e
I20260807 20:12:37.947202 27291 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: loading cluster ID for follower catalog manager: success
I20260807 20:12:37.947973 27291 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: acquiring CA information for follower catalog manager: success
I20260807 20:12:37.948233 27291 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5d93525b095e409a9fe15195622f1123: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:12:37.971087 27306 catalog_manager.cc:1259] Loaded cluster ID: 425ce399867041a29c9f6d6c3f06680e
I20260807 20:12:37.971168 27306 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: loading cluster ID for follower catalog manager: success
I20260807 20:12:37.972033 27306 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: acquiring CA information for follower catalog manager: success
I20260807 20:12:37.972370 27306 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 624c1391b6094af1ad57c9c875236497: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 20:12:38.168529 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:38.165542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.3.15.189:35881)
0807 20:12:38.165742 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:38.165744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:38.165758 (+ 14us) server_negotiation.cc:407] Connection header received
0807 20:12:38.165825 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:38.165825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:38.165916 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:38.166006 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:38.166162 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.166778 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.167532 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.167760 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.167832 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:38.168061 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:38.168216 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:38.168303 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:38.168391 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 20:12:38.168420 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":76,"threads_started":1}
I20260807 20:12:38.172647 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:38.170410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.3.15.188:39895)
0807 20:12:38.170514 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:38.170516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:38.170586 (+ 70us) server_negotiation.cc:407] Connection header received
0807 20:12:38.170650 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:38.170651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:38.170686 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:38.170758 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:38.170841 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.171311 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.171975 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.172173 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.172226 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:38.172344 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:38.172454 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:38.172506 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:38.172540 (+ 34us) server_negotiation.cc:299] Negotiation successful
0807 20:12:38.172556 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":56,"threads_started":1}
I20260807 20:12:38.177276 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:38.174352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43248 (local address 127.3.15.189:35881)
0807 20:12:38.174508 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:38.174510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:38.174636 (+ 126us) server_negotiation.cc:407] Connection header received
0807 20:12:38.174737 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:38.174738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:38.174748 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:38.174778 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:38.174973 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.175422 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.176433 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.176702 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.176733 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:38.176988 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:38.177019 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:38.177073 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:38.177164 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 20:12:38.177189 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260807 20:12:38.177331 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:38.174406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.3.15.190:39887)
0807 20:12:38.174527 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:38.174529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:38.174540 (+ 11us) server_negotiation.cc:407] Connection header received
0807 20:12:38.174648 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:38.174649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:38.174704 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:38.174785 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:38.174984 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.175724 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.176456 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.176762 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.176848 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:38.177005 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:38.177115 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:38.177179 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:38.177204 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 20:12:38.177227 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260807 20:12:38.177398 27414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 20:12:38.174398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.3.15.188:39895)
0807 20:12:38.174439 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 20:12:38.174442 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 20:12:38.174622 (+ 180us) server_negotiation.cc:407] Connection header received
0807 20:12:38.174702 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 20:12:38.174703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 20:12:38.174713 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 20:12:38.174782 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 20:12:38.174878 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.175613 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.176591 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 20:12:38.176912 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 20:12:38.176975 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 20:12:38.177137 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 20:12:38.177172 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 20:12:38.177219 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0807 20:12:38.177310 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 20:12:38.177333 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 20:12:38.179045 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 27046
I20260807 20:12:38.189672 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 27125
I20260807 20:12:38.195073 3134 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskp_X5oZ/build/release/bin/kudu with pid 27205
2026-08-07T20:12:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2971 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5806 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (276280 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST